With the emergence of privacy laws and regulations such as GDPR and CCPA, concerns have been shifted to their compliance, especially for today's mobile apps. A key challenge in mobile privacy compliance is that mobile SDKs' ⇤ The two lead authors contributed equally to this work. † Corresponding authors. data practices are often unclear to apps, yet privacy laws hold app developers accountable [36,68,80,83]. Recently, to help apps comply with privacy laws and regulations, many SDKs provide mechanisms, particularly privacy APIs, for apps to configure the SDK's data practices (e.g., on data collection, sharing, or selling) to comply with privacy laws. For instance, for compliance with the Children's Online Privacy Protection Rule (COPPA), AppLovin's SDK [22], which is an ad network, offers the privacy API setIsAgeRestrictedUser. Once this API is invoked by an app, the SDK will refrain from collecting and processing any data from the current (child) user [15]. While different SDKs support various privacy APIs for apps to configure their compliance with specific laws, such a paradigm, dubbed privacy-configurable SDKs (or PICO SDKs), has been implemented by a large portion of top SDKs and is widely used in apps on the Android platform.
Emerging privacy risks with PICO SDKs. The PICO SDKs are supposed to improve privacy assurance for Android apps. However, our preliminary study reveals subtle privacy compliance risks in (1) how apps use the privacy configuration APIs (or capabilities), and (2) how PICO SDKs implement and fulfill the expected compliance. Essentially, the PICO SDKs and their adoption bring a variety of general and novel privacy compliance risks, collectively called privacy-configuration compliance risks or PICO risks in our research. While observations of noncompliance related to SDKs have been reported before [51,71], to the best of our knowledge, there has been no systematic, large-scale study on the PICO risks to understand the fundamental causes of the problem.
Particularly, we found that configuring privacy compliance through PICO SDK can be more complicated than expected. An app often uses multiple PICO SDKs (e.g., for serving ads, analytics, and social networks) and is supposed to properly configure relevant privacy APIs of all these PICO SDKs. A PICO SDK, such as the ads network SDK AppLovin, internally wraps and uses ads from other ads network SDKs, such as Google AdMob and Unity Ads, which are also PICO SDKs with privacy APIs. These PICO SDKs wrapped by AppLovin are transitive dependencies of the app and we call AppLovin a PICO wrapper. The wrappers and the SDKs they wrap form complicated delegation relations in privacy configurations: once the app configures specific compliance requirements for the wrapper (e.g., age restriction), the wrapper is supposed to faithfully propagate the privacy configurations to all PICO SDKs it encapsulates. This can easily go wrong, leading to privacy risks. Specifically, in the absence of standards, not all (wrapped) PICO SDKs support the same sets of regulations, and for the same regulations, they may not even come with the same levels of support. Privacy violations have actually occurred, for example, when an app configures AppLovin for GDPR compliance, while such a configuration cannot be propagated to the wrapped SDK Google AdMob, due to the latter's partial support for GDPR. In the paper, we present detailed case studies and measurements of PICO risks ( § 6), together with nine different types of PICO risks ( § 4).
Detecting privacy risks with PICO SDKs. We developed PICOSCAN, an automatic privacy risk analysis framework, to detect PICO risks in real apps on Android, one of the most common mobile platforms. This is based on a set of PICO privacy principles we summarized for different roles in the PICO ecosystem, i.e., the host app, PICO SDK and PICO wrapper ( § 3.3). PICOSCAN leverages a PICO usage inspector to analyze the use of PICO SDKs in the apps to determine whether the use violates the app-related principles. It also utilizes a PICO SDK inspector to check whether individual and wrapper PICO SDK implementations uphold applicable privacy principles. These two components were developed to mainly perform static analysis, with supplemental dynamic analysis to support PICO risk detection for specific PICO privacy principles ( § 4).
Measurement of PICO risks in the wild. To understand the compliance risks associated with PICO SDKs in the wild, we built a PICO SDK metadata database named PICO METADB. This database comprises 65 of the most popular PICO SDKs on the Android platform, covering three major categories: 43 ad SDKs, 20 analytics SDKs, and 2 social network SDKs. Running PICOSCAN on 48,305 Google Play apps (downloaded in October, 2023), our measurement results indicate that despite the enhanced privacy configurability, PICO SDKs and their use in the wild are far from being compliant, and privacy risks can occur at every step of SDK integration and be introduced by every role in the PICO SDK ecosystem. Overall, among 24,844 apps in our dataset that integrate PICO SDKs, 7,880 apps (31.7%) are found to contain at least one PICO risk. Specifically, 31.5% apps fail to invoke privacy APIs in the first place, leaving privacy configurations of PICO SDKs unattended. The risk is substantial considering that over 43.1% PICO SDKs include privacy-invading default configurations that allow the collection and processing of user personal data. Even for apps that invoke privacy APIs, they are likely to configure the APIs in a way inconsistent with users' preferences, e.g., 259 apps either hard-code privacy APIs with data-enabling values directly, or set privacy APIs in conflict with user privacy preferences. Further, we found that there are numerous privacy risks even for the apps that perfectly configure privacy APIs, due to the erroneous implementations in PICO SDKs and PICO wrappers. For example, 16 PICO SDKs that cover 14,869 apps have ambiguous consent configurations such that they still collect user personal data even when privacy APIs are invoked to decline the consent. Things become even more complicated for PICO wrapper: 12 PICO wrappers (54.5% out of 22) are found to not relay privacy configurations to the PICO SDKs they encapsulate.
These findings shed new light on PICO SDKs, leading to better understanding about their privacy assurance. First, with the modularization of app development using closed-source components from diverse parties, developers require enhanced tooling, including privacy-configurable SDKs, to maintain consistent privacy promises across all app components. However, this study demonstrates that this goal may not be easily achieved solely relying on configurability. Inconsistent privacy configurations may arise from various development flaws (e.g., missing or erroneous configurations), and unfulfilled responsibilities by PICO wrappers (e.g., broken relay). Second, the design and use pattern of PICO SDKs tend to expose app users to privacy risks. Third, misimplementation and misinterpretation are common in PICO SDKs, possibly due to the absence of systematic review and proper standards. Based on our findings, we provide suggestions to key stakeholders, such as developers of apps and PICO SDKs, as well as policy makers and law enforcement agencies, with the hope of helping end-users obtain genuine privacy assurance when using Android apps ( § 7).
• We present a systematic study of privacy risks for the use and implementation of privacy-configurable SDKs (PICO SDK) on Android, and identified a range of general, novel privacy compliance risks, collectively called PICO risks.
• We developed PICOSCAN, a compliance risk analysis framework to automatically identify PICO risks in Android apps and PICO SDKs. We will fully release the source code.
• We performed a large-scale study of PICO risk with over 48 thousand Google Play apps and 65 popular PICO SDKs. This reveals that PICO SDKs, though created for privacy compliance, often fail to provide expected privacy assurance.
Privacy configuration capabilities supported by thirdparty SDKs. The mobile SDK ecosystem has evolved significantly, with many SDKs now offering dedicated "advanced privacy settings" documents (such as [29]) to assist app developers in achieving privacy compliance. These documents often contain three types of information: legal and privacy disclosure, app-level privacy configurations, and user-level privacy configurations. Legal and privacy disclosure outlines privacy policies describing how an SDK processes and shares information, along with methods for end users to exercise privacy rights (e.g., through data subject access request). App-level privacy configurations enable app developers to customize SDK usage and functionality mostly from the server side (e.g., via SDK dashboard), such as specifying data points for an analytics SDK. These configurations apply to all end-users of an app. User-level privacy configurations enable app developers to customize privacy exposure for each end-user. This study focuses on user-level privacy configurations.
Privacy Configurable (PICO) SDKs. To support user-level configurations, third-party SDKs may offer a set of predefined privacy APIs. We refer to such SDKs as privacy-configurable SDKs (PICO SDKs). The privacy APIs of PICO SDKs may control various data practices, such as data collection, sharing, and selling, in compliance with privacy laws (e.g., GDPR). When using a PICO SDK, an app often invokes its initialization API to configure global SDK options. For example, an ads SDK initializes the ads unit ID before requesting ads from backend ad networks. PICO SDK privacy APIs are typically invoked alongside the SDK initialization APIs to ensure that privacy configurations are picked up early on by SDKs.
3 Overview of PICO SDKs
Landscape of PICO SDKs. Privacy APIs of PICO SDKs aim to help apps fulfill requirements of a range of privacy laws, regulations, and industry standards, such as GDPR [26], COPPA [25], US states' consumer privacy acts [27,46,75,76], China's Personal Information Protection Law (PIPL) [30], and CARU Advertising Guidelines [24]. We conducted a survey to understand the privacy laws supported by popular Android SDKs. Specifically, we gathered a list of 203 most popular SDKs (ranked by AppBrain [4], a service providing Android market statistics) including 137 Ads SDKs, 41 Analytics SDKs, and 25 Social Network SDKs. We focus on the three SDK categories since they were known to involve privacy practices [65,67,86]. We asked two security researchers to find API documents for the popular SDKs, and then cross check and combine their results. In total, we found API documents for 91 of the 203 SDKs. From the documents, we searched privacy APIs based on keywords including "privacy", "compliance", "regulations" and names of specific laws. The most commonly supported laws are GDPR (61 SDKs), CCPA (39 SDKs), and COPPA (34 SDKs), spanning a total of 65 PICO SDKs. We provide a detailed list of privacy laws and regulations along with the number of SDKs that offer privacy APIs to ensure compliance online [17].
Based on the survey, our study focuses on the support of three laws (GDPR, CCPA, COPPA) by the 65 PICO SDKs (see PICO METADB in Section 4.4). We study how effectively their privacy APIs support privacy compliance and whether apps have used them properly to achieve expected compliance goals. Notably, although each privacy law defines a range of privacy-related requirements, we specifically focus on the following requirements for each law, which are most commonly supported by the privacy APIs. We documented all 191 privacy APIs in a database ( § 4.4) and have released them at [17].
• GDPR compliance with consent requirements. Based on GDPR, PICO SDKs (as data controllers) should establish a legal basis for processing personal data, in most cases by obtaining users' consent. Hence, many PICO SDKs have provided privacy APIs with parameters that allow app developers to specify users' consent status, thereby enabling the PICO SDKs to administer private data processing according to the consent status. An example is Vungle [79], an ad network SDK which offers an API setGDPRStatus. By passing in a false parameter value, an app that integrates Vungle can instruct Vungle to avoid collecting a range of personal data restricted by GDPR, such as Android ID and advertising ID.
• CCPA compliance with the right to opt-out. Under CCPA, PICO SDKs may not have to obtain explicit user consent for data collection (although businesses must provide clear and conspicuous privacy notice). Rather, CCPA requires that end users who are California residents be able to opt-out of data sharing and selling from businesses. To fulfill such a privacy right, PICO SDKs commonly provide a privacy API for apps to specify end users' choices of "opt-in" or "opt-out", based on which the PICO SDKs restrict the sharing and trading of user data. For example, the Vungle SDK has an API setCCPAStatus to capture users' "opt-out" status.
• COPPA compliance and protection for children users. COPPA limits the collection and use of children's personal data, except with explicit and verifiable consent from parents. Other laws such as GDPR and CCPA also have corresponding restrictions for processing children's personal data. For compliance, a most common practice we observed from PICO SDKs is to provide a privacy API that allows apps specify whether the current user is under age restriction (being a child). The PICO SDKs will then refrain from collecting data from the child users. For example, the Vungle SDK provides a privacy API (setCOPPAStatus) that takes a boolean parameter to indicate whether the current app user is a child or not.
We show real examples of PICO risks in Figure 1. CallApp [60] 1 is a popular app that has over 100 million downloads on Google Play. It offers attractive security features for users, such as detecting caller identity, and blocking spams. The app, for maximized monetization, implemented in-app ads bidding that selectively displays ads from multiple ads networks using their SDKs, such as Google AdMob, AppLovin, and Facebook Audience Network. Offerings of these ads SDKs include showing personalized ads based on users' personal data (e.g., user and device identifiers, and demographic information, etc.). Most of the ads SDKs are PICO SDKs, providing privacy APIs for apps to configure them in accordance with applicable privacy laws, such that the apps as a whole can be compliant. Upon analyzing the CallApp app, we found that both the in-app use and internal implementations of PICO SDKs are complicated, resulting in several types of subtle privacy compliance risks in Android apps that were unknown, or at least less known before. First, the app intends to be GDPR-compliant by calling AppLovin SDK's privacy API setHasUserConsent with an argument of either true or false, depending on the user choice. AppLovin itself will not collect or use personal data for delivering personalized ads without user consent (GDPR compliance). However, the AppLovin SDK not only delivers ads from its own ads network, but also internally wraps and bids ads from other ads SDKs, such as IronSource, Vungle, Google AdMob, and Unity Ads, etc. These SDKs wrapped by AppLovin are transitive dependencies of the app and we call AppLovin a wrapper SDK or PICO wrapper. The expectation is that, once the app urges a PICO wrapper like AppLovin for GDPR-compliance, it should propagate the compliance requirement to all ads SDKs it internally uses (by calling their privacy APIs for GDPR). We find that the expectation fails to be fulfilled for key challenges: (1) Not all wrapped ads SDKs support compliance configurations evenly. Google AdMob, for example, partially supports GDPR configuration: after a PICO wrapper passes along "not consent" to AdMob's privacy API, AdMob will not deliver personalized ads but will always collect user data including device/user identifiers [1,2]. Failing to account for the different levels of support for the same law among PICO SDKs results in compliance risks for the app. (2) Not all PICO SDKs used by a PICO wrapper support the same laws. For example, the Yandex SDK and Kidoz SDK have not supported privacy configurations for CCPA, and both of them can be used by the wrapper AppLovin.
Second, with many PICO SDKs used in an app, their host app or PICO wrapper might not configure compliance APIs of all PICO SDKs, or configure them in a correct manner. Ogury, for example, is an ads SDK supporting COPPA configuration. However, AppLovin does not call Ogury's COPPA API applyChildPrivacy, although AppLovin did call COPPA configuration APIs of all other ads SDKs it wrapped. Also, for CCPA compliance, when the user is a California resident, the app is supposed to invoke CCPA APIs in PICO SDKs to communicate the user's opt-out status for data sharing/sale. Although the app uses the Facebook Audience Network (SDK) to deliver ads, it uses a hard-coded "opt-in" option when configuring Facebook SDK's CCPA API setDataProcessingOptions, posing a violation to users who choose "opt-out".
Further, although many PICO SDKs feature privacy APIs that support multiple laws, their default configurations (without privacy API calls by the app) do not comply with privacy laws. For example, the InMobi SDK [28] by default does not comply with COPPA unless the app calls the setIsAgeRestricted API with an argument value true.
We cannot evaluate privacy compliance risks without specific privacy principles tailored for PICO SDKs. This section, based on the motivating example ( § 3.2), summarizes a set of such principles for different roles in the PICO ecosystem, i.e., the host app, PICO SDK and PICO wrapper.
Privacy principles for apps using PICO SDKs. Privacy APIs in PICO SDKs are designed to meet the key privacy objective of returning privacy control to end users (in alignment with the expectations reported in prior studies [42,51,64,73]). Hence, it is crucial that apps integrating PICO SDKs configure the privacy APIs correctly and effectively based on the actual privacy preferences of end users.
Principle #1 (Complete, User-controlled Privacy Configuration): Apps should ensure complete, correct privacy configurations of all PICO SDKs in accordance with user preferences.
Privacy principles for PICO SDKs. Similar to the principle of "secure by default" [49], which requires automatic enabling of security measures without user intervention, the principle of "privacy by default" has become common sense among privacy practitioners [37,52,58] and is also part of privacy laws [44,48].
Principle #2 (Privacy by Default): PICO SDKs should have default configurations that protect user privacy.
Individual PICO SDKs offer privacy APIs for enabling apps using them to manage PICO SDK privacy practices. Hence, it is natural to mandate that privacy APIs to effectively fulfill all promised privacy compliance without ambiguity, akin to the "complete mediation" security principle by Saltzer and Schroeder [72].
Principle #3 (Nonambiguous, Fulfilled Privacy Compliance): PICO SDKs should fulfill all promised privacy compliance once relevant privacy APIs are configured; the promise (for the compliance effect of privacy API configurations) should be nonambiguous.
Privacy principles for PICO wrappers. As demonstrated in § 3.2, when using PICO wrappers (one that encapsulates other PICO SDKs), the underlying PICO SDKs are not exposed to apps directly. Instead, their privacy APIs are configured by the PICO wrappers, similar to the delegation design pattern [56]. Therefore, it is the responsibility of the PICO wrappers to properly propagate (or forward) privacy configurations of the app to all internal PICO SDKs.
Notably, a PICO wrapper itself is usually a PICO SDK by providing privacy compliance APIs (e.g., AppLovin in Figure 1) and thus should follow principles of PICO SDKs.
Principle #4 (Complete Privacy Delegation): PICO wrappers should fully propagate app's compliance configurations to all internal PICO SDKs in use.
Discussion. The principles are not direct adoption of prior general principles, and they are contextualized to PICO SDKs. Consequently, the principles will guide the identification of PICO risks in real apps and SDKs ( § 4), although they may not be exhaustive. For example, given that data practices extend into PICO SDK backends, it becomes necessary to establish privacy principles that apply to these backends. We leave further exploration of those principles for future research.
PICO risks occur when apps, PICO SDKs, or PICO wrappers violate the above privacy principles ( § 3.3). In this section, we introduce PICOSCAN, an automatic analysis framework to detect PICO risks.
Overview. Figure 2 shows the overview of PICOSCAN, including the techniques and tools used in it. PICOSCAN takes Android apps as input and identifies PICO risks in two aspects: (1) whether and how the apps use PICO SDKs, and (2) how PICO SDKs and PICO wrappers implement their privacy APIs to meet the expected privacy compliance, with two main components, i.e., PICO usage inspector (PUI) and PICO SDK inspector (PSI), respectively.
PUI first identifies the usage of PICO SDKs in the apps through static analysis, which involves searching for package names of PICO SDKs and invocations of their privacy APIs. This is enabled by a PICO SDK metadata database (PICO METADB) that we built from the 203 most popular mobile SDKs listed on AppBrain [4]. The database contains various useful information for the analysis (as we will detail in § 4.4), such as the package names of PICO SDKs, API signatures, etc. For each privacy API invocation found in the apps, PUI further identifies whether the invocation is made by the app code or by a PICO wrapper. In the case of the former, PUI applies appspecific principles (Principle #1) to detect privacy risks ( § 3.3) related to app code invoking the privacy API. For the latter, PICOSCAN leverages PSI to detect privacy risks based on wrapper-specific principles (Principle #4). PSI further inspects the implementation behind the privacy APIs of PICO SDKs and identifies privacy risks based on PICO SDK related principles (Principle #2 and #3). Note that PSI analyzes the PICO SDKs and PICO wrappers that are already compiled into the apps as an alternative to analyzing separate SDKs, since some SDKs are not publicly available, e.g., Amazon mobile ads SDK [21] is invitation-only.
Directly applying high-level principles in code-level detection is difficult. Thus, we developed a set of more specific, fine-grained technical patterns of PICO risks, called PICO Violation Patterns (PVPs), corresponding to each principle (Table 1). PUI and PSI scan for the occurrences of the PVPs in the app code, the code of PICO SDKs and PICO wrapper hosted in the apps, and report them as PICO risks. PICOSCAN is mainly built on static analysis and is supplemented by dynamic analysis for the scan of certain PVPs. For example, under Principle #2, to identify the default compliance status of PICO SDKs (e.g., true or false for age-restricted data collection), we leverage static analysis to identify specific variables that store the compliance value and then use dynamic analysis to find the exact default value (detailed in § 4.2.2). Figure 2 outlines the static and dynamic analysis techniques and tools adopted for different PVPs. We will elaborate on the choice of these techniques in the latter part of this section. To determine whether an app uses PICO SDKs, including whether privacy APIs are invoked and whether they are invoked by first-party code (app code) or third-party code (PICO wrappers), PUI builds a global call graph for the app and traverses all the edges to identify reachable methods that match PICO SDK initialization APIs, as defined in the PICO METADB. It then determines the parties using the PICO SDKs by comparing the caller of the initialization APIs to the package names of the main app or any other SDKs. The identification of PICO wrappers is slightly difficult, SDKs often lack clear disclosure about whether they are wrappers for other SDKs. In this study, we identify PICO wrappers by uncovering inter-SDK connections on an app's call graph. Specifically, for each discovered PICO SDK initialization API on the call graph, we explore the subgraph rooted in this API to locate the initialization APIs of another SDK. This relationship, where the initialization API of one SDK leads to that of another, is a clear indicator that the former SDK functions as a wrapper for the latter SDK.
Under the privacy principles governing apps (Principle #1), we focus on three PVPs whose presence in the first-party code indicates risks associated with the app's use of privacy APIs 2) users' privacy preferences typically collected through app UIs, such as whether they consent to data collection (see Figure 7 in the Appendix for an example). We elaborate on the PVPs as follows.
Missing configurations (PVP #1). Apps using a PICO SDK but failing to configure the privacy compliance of the SDK by invoking its privacy APIs indicate a PICO risk.
An app often uses multiple PICO SDKs but may not configure all of them. An example is the "Copy My Data" (com.mediamushroom.copymydata) app [59], which leverages the IronSource SDK to show ads but does not invoke any of the IronSource privacy APIs.
Configurations disregarding or inconsistent with user privacy preferences (PVP #2). Apps are expected to capture user privacy preferences and communicate them to PICO SDKs by properly invoking the PICO SDKs' privacy APIs. However, they might fail to capture user privacy preferences or, if they do, they may fail to use the proper parameter values when invoking the privacy APIs. A special situation within this pattern is when an app uses hard-coded parameter values (e.g., consent: true) to configure privacy APIs -an issue happened in many apps. Concerning COPPA compliance, for example, the "SuperLozzi" (com.ltlk.z7) app, a play-and-earn app with over one million downloads on Google Play, collects the user's date of birth. However, the app calls AppLovin's COPPA API setIsAgeRestrictedUser with a hard-coded parameter value of "false", which disables Ap-pLovin's COPPA compliance even when the user is a child (see measurements in § 6.2).
Erroneous configurations (PVP #3). Even if apps configure the necessary privacy APIs with the correct parameters that reflect user preferences, their configurations may not effectively achieve the desired compliance. This happens when the use of privacy APIs fails to meet the programming requirements specified for using these APIs. For example, the "Solitaire King" (com.solitaire.king.card.creative.wuliu) app incorporates the PICO SDK AppLovin to display ads and configures multiple SDK privacy APIs related to CCPA, GDPR, and COPPA. However, AppLovin requires its initialization function, initializeSdk, to be invoked after configuring its privacy APIs. Unfortunately, this requirement was not followed by many apps, rendering their privacy configurations ineffective.
To detect missing configurations (PVP #1), PUI traverses the apps' call graphs to find out whether the apps initialize and use a PICO SDK but do not call any of their privacy APIs.
For PVP #2, we detect the inconsistencies between privacy configurations and the user's privacy preferences using dynamic analysis. Specifically, PUI launches the apps under analysis, exercises their UIs, and utilizes accessibility service (AS) APIs [20] to monitor the rendered app UIs. PUI identifies privacy consent dialogs that request user consent for data collection and usage (Figure 7 in the Appendix). This is done by matching a set of common privacy-related keywords in the texts of consent dialogs, i.e., via regex matching "ˆ.*(?:privacy policy|collect your|i agree|your consent).*$". PUI leverages AS APIs to automatically click the button on consent dialogs to decline the consent request, by matching keywords such as "no", "decline", "cancel", "not now" or "disagree". Meanwhile, PUI instruments and monitors runtime parameter values of PICO SDK privacy APIs, in particular those indicating "true" or "false" for user consent, and "opt-in" or "opt-out" for data sharing and sale. Since the consent request has been declined, runtime values indicating a true" for user consent or opt-in" represent privacy risks related to PVP #2. PUI uses Frida [8] for monitoring runtime values. In case that privacy consent dialogs are not observed, PUI performs constant propagation on the interprocedural control-flow graph (ICFG) of the apps to determine whether the parameters of privacy APIs are constants (i.e., hard-coded).
Under PVP #3, PUI checks whether invoking individual privacy APIs must occur before or after invoking other APIs of the PICO SDKs. In our study, the ordering requirements were extracted from SDK documents and recorded in the PICO METADB (14 privacy APIs from eight PICO SDKs require specific orders, see Appendix 9). Using the PICO METADB, PUI checks the app code and API calls along the control flows to identify misordered API invocations.
PSI analyzes individual PICO SDKs to check violations to related principles (principles #2 and #3 in § 3.3), focusing specifically on three PVPs as elaborated below.
Privacy-invading by default (PVP #4). The PICO Principle #2 requires that individual PICO SDKs come with a default configuration in compliance with related privacy regulations. However, this is not true for many PICO SDKs. Take, for example, the SDK of InMobi [28], a mobile marketing platform specializing in personalized ads. The InMobi SDK will stop collecting certain personal information (such as advertising ID) and notify its server to comply with COPPA for the current user only after the host app calls the setIsAgeRestricted API with a true parameter value. If the above API is not called, InMobi defaults to assuming that all app users are adults, to whom COPPA does not apply, and thus allows for the collection of personal data for user profiling and serving personalized ads. Privacy-invasive default configurations like these are found in many popular PICO SDKs (see measurements in § 6.3).
Ambiguous consent configuration (PVP #5). At the core of many privacy regulations is a requirement for user consent before processing personal data. For example, GDPR requires that data controllers (apps or SDK providers) must obtain user consent for the collection and use of personal data, including the use for serving personalized ads [26]. Under this requirement, the requestConsentInfoUpdate API of AdMob is used to help apps collect user consent "to use your personal data for personalized ads" (by showing a consent dialog developed by the SDK, Figure 6), and apply the user consent status for the SDK's operation. While not all PICO SDKs come with built-in consent dialogs, they commonly provide APIs for apps to specify the user's consent (true or false), such as for personalized ads [39]. However, the intended compliance effects of the consent status can be ambiguous: for configurations of consent: false, it is unclear whether the SDKs will not collect user personal data, or they will still collect user data but just do not use the data for specific purposes such as serving personalized ads. Our study of real PICO SDKs showed that such an ambiguity is indeed underlined by different compliance effects among PICO SDKs, resulting in privacy risks: for example, the AdMob SDK always collects user identifiers such as adid and ip address regardless of the consent status configured by the API; in contrast, Kochava will disable the data collection being more compliant.
Ineffective configuration (PVP #6). When apps correctly configure privacy APIs, the PICO SDKs are expected to achieve the compliance goals. Problems arise when the PICO SDKs have implementation mistakes that render privacy APIs ineffective. For example, the MyTarget SDK offers a GDPR API setUserConsent to support the configuration of user consent for data collection. Behind the API, the user consent status is passed along and stored in an SDK internal variable isConsent. In the SDK implementation, every collection of device identifiers, such as Google and Huawei ad IDs, occurs after inspecting the variable. Such a variable (referred to as IPSVs in PICOSCAN's detection, § 4.2.2), essentially records the configured consent status, which is important information governing the SDK's privacy practices. However, in the MyTarget SDK implementation, the isConsent variable can be modified by other unexpected program paths, specifically from the SDK's CCPA API setCcpaUserConsent. Any consent status configured for CCPA will overwrite that of GDPR, silently rendering the GDPR configuration ineffective. This overwriting causes problems in cases where the apps integrating the SDKs need to ensure compliance with both GDPR and CCPA, e.g., when an app must protect users who are California residents while the app itself operates as an EU-based business.
Detecting default privacy-invading configurations (PVP #4). To assess whether the privacy configurations of a PICO SDK are privacy-invading by default, we developed a method in PSI to model the implementation of the PICO SDK's configurations. Specifically, our observation is that once a privacy API is invoked with certain parameters (called privacy parameters), a PICO SDK usually stores the values of the parameters (e.g., whether the user is a child) as internal states in local variables, such as class or instance fields. We refer to such variables storing privacy parameter values as internal privacy state variables or IPSVs. Further privacy practices of the PICO SDK, such as invocations of system APIs to collect personal data, will be governed by checking the IPSV values. In the InMobi SDK, for example, the data flow behind its COPPA API setIsAgeRestricted shows that the privacy parameter isAgeRestricted is eventually stored in an IPSV, i.e., <com.inmobi.media.ea: Boolean c>. In multiple SDK functions that attempt to collect user personal data, it first checks the IPSV to determine whether the collection should proceed. Notably, while the privacy parameter can be stored in various locations, including in shared preferences [31] for persistence, it is often loaded into the IPSV for the SDK to easily access it in different functions.
Based on the above observation, PSI first performs static taint analysis on the apps to identify the IPSVs corresponding to the SDK's privacy parameters. This analysis is then complemented by dynamic analysis to identify the runtime values of these IPSVs using Frida [8]. If the runtime values of the IPSVs match the values enabling data collection, sharing or selling as recorded in the PICO METADB, PSI reports a PICO risk corresponding to PVP #4. More specifically, to identify IPSVs, we use privacy APIs' parameters as sources and find program variables to which the parameter values are propagated (within the SDK code). Considering that the values may be propagated to some intermediate variables, PSI filters out those with different types from the privacy parameters and those local variables internal to a method; PSI specifically retains variables used in a condition check that acts as a guard for user personal data collection.
Identifying ambiguous consents (PVP #5). To detect PVP #5, PSI identifies the collection of personal data in the SDK code even after configuring consent: false in the invocation of privacy APIs. The first step uses static analysis: PSI identifies code locations that access personal data. This is done by checking invocations of a list of system APIs (compiled from prior work [70,87]) that access device and user data (e.g., device identifiers, advertising identifiers, and location). Similar to prior work [40], the static analysis also finds out the program paths in the SDK that transmit the user data to the Internet. Next, PSI runs the app and uses consent: false to configure the privacy APIs of PICO SDKs. This is done using dynamic instrumentation that manipulates related privacy parameters upon the app's invocation of privacy APIs. PSI reports privacy risks if it finds that the program paths that collect and transmit user data are executed at runtime (by inspecting call stack traces), even under consent: false configurations of the privacy APIs.
Identifying ineffective configurations (PVP #6). The presence of multiple program paths manipulating common IPSVs related to privacy APIs is an indicator of potentially ineffective configurations. Therefore, we detect them by checking whether other public APIs of a PICO SDK can cause changes to the IPSVs of any of its privacy APIs. For this purpose, we extract the set of IPSVs, S IPSV , for each privacy API of a PICO SDK. Next, we perform static taint analysis to collect all internal variables affected by other public APIs of the same PICO SDK, i.e., S v . We cross-compare the S IPSV and S v to identify whether the IPSVs of the privacy API are manipulated by other program paths. Using this method, we can detect ineffective configurations in other 8 PICO SDKs besides MyTarget, such as Appodeal, and BidMachine (see measurements in §6.3).
The PICO wrapper SDKs are special PICO SDKs that provide privacy APIs to apps and are expected to fully and correctly propagate apps' privacy configurations to their wrapped PICO SDKs. This requirement is instrumental considering that the wrapped PICO SDKs are transitive dependencies of the apps, which are often unknown and inaccessible for app developers to configure. Under the Principle #4, PICOSCAN focuses on three patterns of violations related to PICO wrappers.
Uneven privacy support (PVP #7). In the absence of standards for PICO SDKs, individual PICO SDKs may support different sets of laws and regulations, and for the same regulations, they may not even come with the same levels of support. For example, PICO wrappers Appodeal [38] and AppLovin [47] provided privacy APIs for GDPR compliance. Meanwhile, they both wrapped Meta Audience Network [7], SDK of another ads network, to help serve ads. Notably, Meta Audience Network has not provided privacy APIs to configure GDPR compliance. This can directly lead to privacy risks with GDPR in real apps. For example, when the app com.solutions.class11dkg configures Appodeal with "consent: false" for data collection based on the user's privacy preference, Appodeal cannot propagate the configuration to Meta Audience Network, which actually collects user data and serve ads in the app regardless of user's consent, posing a risk.
Broken privacy delegation (PVP #8). Even when the PICO wrapper uses an individual PICO SDK that has aligned privacy law support, it is possible that the PICO wrapper fails to fully propagate the app's privacy configurations to the corresponding APIs of the individual PICO SDK. For example, AppLovin (PICO wrapper) internally uses the Facebook Audience Network (PICO SDK) for serving ads. The AppLovin SDK has a CCPA API setDoNotSell, that enables app users to opt out of data sharing/sale using a true parameter. However, the AppLovin SDK does not use such a parameter or related information to invoke the Facebook CCPA API AdSettings.setDataProcessingOptions, leading to a privacy risk.
. A variant of the broken delegation is the presence of extra conditions imposed by the wrapper that blocks propagation of privacy configurations. An example is the PICO wrapper Admost and its wrapped SDK Unity Ads. Both SDKs have a GDPR related API to configure user consent for data collection, i.e., AdMostConfiguration.setUserConsent, and MetaData.set respectively. Based on user choices, when the host app configures Admost with the user's consent status for collecting personal data, Admost will only conditionally propagate the consent status to Unity Ads. When Admost considers the user's IP address is not within EU countries (thus may not be subject to GDPR), it will configure Unity Ads with a "consent: true". This happens even if (1) the user does not consent to data collection or (2) the app intends to configure the PICO wrapper not to collect user data. Such a design pattern is problematic: (1) it can violate the app's compliance expectation and privacy goal; (2) using IP addresses to determine whether the user is subject to GDPR is not reliable [6,12]. Further, such a practice violates Unity Ads' privacy goals for users: Unity Ads noted that "Consent extends beyond GDPR and should be applied in any region" [77].
PUI detects PVP #7 by comparing privacy APIs and different privacy laws supported by the wrapper and its internal PICO SDKs. Such information has been recorded in the PICO METADB (see PICO METADB in § 4.4). For PVP #8, PSI adopted static taint analysis to check whether the privacy parameters in the wrapper's privacy APIs are propagated through program paths to parameters of internal PICO SDKs' privacy APIs. PUI considers that such program paths may come with condition checks before invocations of the internal PICO SDK's privacy APIs. To identify improper conditions that restrict the privacy propagation (PVP #9), we leverage an approach based on the data dependency of the conditions being checked, inspired by prior studies that detect hidden operations [84] or logic bombs [88]. Specifically, PSI performs backward data dependency analysis for a condition variable. If its value is independent of a privacy parameter (those in the wrapper's privacy APIs), this indicates that such a condition can be implicit and unknown to the apps that configure the wrapper's APIs. We consider such a condition to be a violation under PVP #9. For example, in identifying the above example involving Admost which imposes an improper condition by checking IP addresses, PSI finds that IP addresses is not propagated from any parameters of Admost SDK's APIs.
PICOSCAN comes with a PICO METADB that stores metadata for 65 PICO SDKs on Android whose privacy APIs support GDPR, CCPA, or COPPA. The 65 PICO SDKs are filtered from 203 popular SDKs (see § 3.1), and cover three major SDK categories: 43 ads SDKs, 20 analytics SDKs, and 2 social network SDKs. Table 2 provides a more detailed breakdown of SDK types and the supported privacy laws.
From the API documents of the PICO SDKs found in § 3.1, we asked the two security researchers to independently identify two types of APIs: privacy configuration APIs and the SDKs' initialization APIs, by searching for keywords, i.e., "initialize", "privacy", and relevant laws including "GDPR", "COPPA", "CCPA", and "US State laws", in the documents. They examined the API descriptions to find indicators (such as "before", "after", and "order") for the invocation order and recorded these invocation order requirements. Afterwards, they reviewed the API arguments to identify which arguments are related to "consent", "opt-out", and "children" (corresponding to three requirements noted in the scope of research in § 3.1). They were then asked to validate each other's findings, resolve any disagreements through discussions, and combine them.
In total, PICO METADB includes 191 privacy APIs and 146 initialization APIs of the 65 PICO SDKs. The privacy APIs include 106 APIs for GDPR compliance provided by 61 SDKs, 47 APIs for CCPA compliance provided by 39 SDKs, and 38 APIs for COPPA compliance provided by 34 SDKs. The PICO METADB recorded metadata of the APIs useful for our analysis, including API signatures (including class names, method names, parameters types), corresponding privacy laws, parameters with privacy semantics and values of compliance (e.g., a true parameter indicating child users in a COPPA API setIsAgeRestricted), and constraints for the correct use of the APIs (e.g., order of API invocations), etc. We show an example of the metadata for Bytedance APIs in Appendix (Figure 5). For reproducibility, the API documents and the resulting PICO METADB have been released at [17].
5 Implementation and Evaluation
The static analysis techniques are mostly implemented with Soot [32], one of the most commonly used frameworks for Java and Android analysis. To achieve better coverage, we built app call graphs using an extensive list of app entries, including all lifecycle functions of the registered app components, as well as app methods that match any of the 879 subsignatures of the system callback methods (as reported in EdgeMiner [45]). We built the Interprocedural Control-Flow Graph (ICFG) in the Soot framework using Heros [16] . On top of the call graph and the ICFG, we implemented techniques for constant propagation, and data dependency analysis. In total, we implemented the static analysis with 4,587 lines of code.
The detection of certain PVPs, i.e., #4, #5, #6, and #8, relies on taint analysis, for which we leveraged FlowDroid [40], a state-of-the-art tool for static taint analysis. Specifically, for PVPs #4 and #6, we used FlowDroid to trace the propagation from privacy API parameters to SDK internal variables (IPSVs) storing the parameter values. A key challenge in directly applying FlowDroid is that it only accepts APIs as taint sources and sinks, rather than specific privacy API parameters. We resolved this issue by adapting FlowDroid's source-sink manager, AccessPathBasedSourceSinkManager, to allow any program statement and variables, including API parameters, to serve as a source in taint tracking. Based on that, we executed FlowDroid using the privacy API parameters recorded in PICO METADB as sources and 138 networking and file access APIs collected from prior studies [66,70,87] as sinks. Then, we identified the IPSVs by analyzing the propagation of privacy API parameters within PICO SDKs through a TaintPropagationHandler. For PVP #5, we track the exposure of user personal data to the Internet by running FlowDroid with a compiled list of 26 sources and 138 sinks based on prior studies [66,70,87]. For PVP #8 (broken privacy delegation), we used the privacy parameters in the PICO wrapper's privacy APIs as sources and the privacy APIs of the encapsulated PICO SDKs as sinks. We report a PVP #8 risk if an expected taint path (or information flow) was not observed.
To perform runtime user preference analysis, we monitor the content on app user interfaces by intercepting all window content change events using the accessibility service (AS) event TYPE_WINDOW_CONTENT_CHANGED. Upon each event, we enumerate all the UI elements by traversing the tree of AccessibilityNodeInfo objects, and tell whether the elements are related to privacy by matching crafted regex (as noted above). We exercise the privacy-related UI elements by issuing automated clicks by AccessibilityNodeInfo.performAction. In addition, the runtime value analysis is built upon Frida [8] -one of the most popular app dynamic instrumentation tools for Android. The dynamic analysis is implemented with 804 lines of Java code and 1,094 lines of Python code.
Groundtruth datasets. To assess the effectiveness of PICOSCAN in detecting PICO risks, we constructed two groundtruth datasets: the app evaluation dataset and the PICO METADB evaluation dataset.
The app evaluation dataset contains 40 randomly selected apps along with their PICO risks related to app-related PVPs (#1-#3). We installed these apps on a Pixel 6 device, manually exercised the apps, and used Frida to record the stack traces and parameters of any PICO SDK initialization and privacy APIs invoked by the apps [8]. To confirm the PICO risks of these apps, we manually checked the stack traces and parameters based on the definition of the app-related PVPs. For example, we flagged PVP #1 if a PICO SDK initialization API was invoked by the app's first-party code without its privacy APIs. The appearance of any PVP in the apps is considered an instance of PICO risk, and in total, we identified 14 instances of PICO risk across 13 apps.
The PICO SDK evaluation dataset includes 10 randomly sampled PICO SDKs from the total of 65, with three of them being PICO wrappers. To execute the SDKs, we selected the app with the highest number of downloads among all apps in APPSET that utilize each SDK. We installed, ran, and analyzed these apps containing the PICO SDKs using similar techniques as described above. We tailored the manual confirmation process to verify the presence of PICO risks related to PICO SDK and PICO wrapper. For instance, to detect PVP #4, we manually identified methods in the apps that return privacy configurations through reverse-engineering, and then hooked into these methods to retrieve the default configuration." This process, particularly the manual reverse engineering and PVP confirmation, was labor-intensive and required two security researchers nine days to complete. In total, we identified 16 instances of PICO risk in these apps. Table 3 provides the breakdown of all the PVPs in the two groundtruth datasets. • False negatives. PICOSCAN reported four false negatives, with two related to PVP #1 and two related to PVP #4. The causes are twofold. First, for PVP #1, PICOSCAN failed to identify missing privacy API invocations in Unity3D. Specifically, Unity3D employs the same API for three different laws, with the first API parameter specifying which law. Due to the limitation that PICOSCAN does not differentiate APIs based on parameters, it mistakenly considers privacy APIs for CCPA and COPPA to be invoked, even though the API is intended solely for configuring GDPR. Second, for PVP #4, PICOSCAN detects the default value of privacy APIs by monitoring the values of IPSVs identified through static analysis. However, certain SDKs, such as Ogury, do not utilize IPSVs for storing privacy configurations; rather, they store the configuration directly into persistent storage (i.e., "IS_CHILD_UNDER_COPPA" in shared preferences). Consequently, PICOSCAN was unable to determine the default value for the absence of IPSVs.
For measuring PICO risks in the wild, we collect a set of 48,305 mobile apps (called APPSET) by crawling Google Play [57] using a Go-based crawler [33] in October 2023. Specifically, we collected the package names of all 7,753,757 apps that have ever appeared on Google Play (as recorded by AndroZoo [34]), and randomly shuffled the package names before crawling. We followed the order of the shuffled package names and crawled an app if it was relatively new (updated after January 1, 2020) and available in both the United States (including California) and the EU 2 . Among the apps in the APPSET, at least 827 were expected to comply with COPPA since they are committed to follow the Play Families Policy [9]. We report GDPR and CCPA risks over the entire APPSET, and COPPA risks over the 827 apps.
Usage of PICO SDKs. Running PUI on the APPSET, we found that 58 PICO SDKs in the PICO METADB are actively used by apps in the APPSET, i.e., SDK initialization APIs are reachable from app entry points. These SDKs appear in a total of 24,844 (51.4%) apps, with AdMob appearing in the highest number (14,497) of apps and AppNexus in the lowest number (3) of apps. It is worth noting that not all apps use the most up-todate SDK version. In particular, 12,480 (50.2%) of the 24,844 apps invoke the legacy version of at least one PICO SDK that does not implement the corresponding privacy APIs. Also, we identified a total of 22 PICO wrappers from the apps using any PICO SDK, and these PICO wrappers are used by 1,769 apps in the APPSET. Each PICO wrapper encapsulates an average of 3.08 SDKs, with AppLovin having the maximum number of 13 SDKs in a single app. An app can also use multiple PICO wrappers. For example, com.zloong.eu.dr.gp app contains nine PICO wrappers, i.e., AdMob, Ironsource and AppLovin, etc.
Overall PICO risks. Table 4 shows the PICO risks that violate privacy principles for three roles, i.e., apps, PICO SDKs, and PICO wrappers. Specifically, at least 7,880 apps (31.7% of 24,844) fail to adhere to the privacy principles for apps and are affected by at least one of PVP #1 to PVP #3. The majority of the apps are affected by PVP #1, missing privacy configuration. Among the PICO SDKs in PICO METADB, 36 violate the privacy principles for individual PICO SDKs and are affected by at least one of PVP #4 to PVP #6. Most (28) of the 36 PICO SDKs use a privacy-invading default value for at least one of their privacy APIs (i.e., PVP #4). Additionally, out of the 22 PICO wrappers, 16 are affected by at least one of PVP #7 to PVP #9, thus violating the complete privacy delegation principle.
Missing privacy configuration (PVP #1). Among the 24,844 apps using any PICO SDK, 7,818 (31.5%) missed privacy configurations, i.e., they use PICO SDKs but fail to set up their privacy APIs. The top 40 apps with the most missed privacy APIs lack, on average, 15 privacy APIs across at least eight PICO SDKs. These apps include very popular ones installed by over 10 million users, such as com.thinkyeah.smartlockfree, which lacks 24 privacy API calls in 10 PICO SDKs. We 2 App availability in the EU is determined by a Spain IP. also found for 16 PICO SDKs, over 50% of the apps using each of them are affected by missing privacy configurations. Notable examples include Criteo (99.2%), Amplitude (94.3%), Bytedance (79.0%), and AppLovin (50.2%).
By examining the distribution of the affected apps and the developer documentations of the 39 affected PICO SDKs involving PVP #1, we were able to infer a few potential reasons. First, we found that the way in which PICO SDKs present privacy configuration options or APIs to app developers is a potential key factor for missing configurations. A large portion (25 out of 39, 64.1%) of PICO SDKs document privacy configuration APIs as advanced and optional features (in contrast to regular steps for SDK integration, see below), up to individual app developers to identify and adopt such advanced features. In contrast, documentations of the other 14 PICO SDKs (35.9%) suggest adoption of their privacy APIs as part of the regular steps for integrating the SDKs. Although adoption of their privacy APIs are still not mandatory, they may better encourage apps developers to identify and use privacy APIs. The latter group of PICO SDKs is significantly less affected by missing configurations compared to the former group, with a Q3 of 64.3% for the former group (i.e., a quarter of the SDKs have a missing configuration rate of over 64.3%) and a Q3 of 32.9% for the latter group, and average missing configuration rates of 43.2% and 23.8%, respectively. We released the list of the affected PICO SDKs in the above two groups, their documents and distribution of the percentage of apps missing privacy configurations for PICO SDKs at the time of our study online [17]. Second, in addition to documents, PICO SDKs also provide different levels of tooling for app developers to correctly integrate them. Take the Criteo and AppLovin SDK as an example. AppLovin provides a mediation debugger that help app developers to visually debug problematic integration of the SDKs, including configurations by privacy APIs, while Criteo does not provide similar tooling. This could be one potential reason why apps using AppLovin are much less affected than those using Criteo. Third, from the apps' perspective, it might be the case that more popular apps may have greater resources for privacy related engineering and testing. We noticed that only a small fraction (4.7%) of apps using Criteo had more than 1,000,000 installations, while a larger proportion (27.9%) of apps using AppLovin achieved such a milestone. Overall, although many PICO SDKs provide privacy configurability, a substantial number of Android apps have yet to adopt them. Hence, the Android ecosystem could greatly benefit from more effective tools, such as those provided by SDK providers, to ensure developers' adoption of these privacy APIs when integrating PICO SDKs.
Notably, we don't have ground truth for what the app developers experienced and what development tools they actually used when integrating PICO SDKs. Hence, we may not judge all reasons why app developers came with PVP #1. However, based on our study, the reasons we discussed above could at least partially cause PVP #1 or make it difficult to avoid in the wild. The similar is true for our causal analysis of other PVPs.
Configuration disregarding or inconsistent with user privacy preference (PVP #2). We found that 259 apps, while invoking privacy APIs, fail to configure the APIs correctly according to user preferences. For example, 248 apps choose to hard-code the parameters of privacy APIs (in six PICO SDKs) which allows unexpected data exposure regardless of their preferences. We suspect that this choice is driven by factors such as the revenues and privacy engineering cost. For example, many apps often serve as ad publishers, which rely on data collection and personalized ads features to maximize revenues. Hard-coding privacy APIs allows them to quickly enable such features with minimal privacy engineering efforts.
In addition, around 8.8% apps using PICO SDKs are found to collect user privacy preferences through privacy dialogs. We found that 11 of these apps still invoke privacy APIs with parameters that enable data exposure, even when the privacy dialogs are declined. A closer inspection of these apps reveals that such discrepancies may stem from app developers' limited knowledge of privacy compliance. For example, the socksrevive.plugin.dragon app displays a privacy consent dialog stating that "By agreeing you are confirming that you are over the age of 16 and would like to personalize your ad experience" and asks users to consent by clicking a "YES, I AGREE" button. The presence of the dialog indicates that the app developer is at least aware of the importance of requesting privacy consent from users. However, the app only uses the dialog as a gating mechanism for new user registrations but fails to propagate the consent status to any other app components, including mobile ads, such as the BidMachine SDK. This problem potentially suggests that the app developer may assume that their app is privacy-compliant as long as a dialog is provided, without fully understanding the necessity of supporting compliance by configuring each individual component.
Erroneous configuration (PVP #3). We observed that eight PICO SDKs, such as AppLovin [23] and Ironsource [29], in the PICO METADB require that privacy APIs and initialization APIs be called in a specific order, which if violated leads to erroneous and invalid privacy configurations. Although the percentage is not high, there are still 56 apps that violate this requirement on three SDKs: AdMob, Ironsource, and Unity Ads. In total, these apps were installed over 20 million times. Given that most apps using these three SDKs meet the requirement, we believe that the issue with the 56 apps is primarily caused by implementation flaws of their developers, considering that all eight of these PICO SDKs explicitly noted the order constraints in their documents. We released the list of the eight SDKs and their documents online [17].
Privacy-invading by default (PVP #4). Among the 65 PICO SDKs, 28 (43.1%) are found to have at least one privacy API with a privacy-invading default configuration, affecting 20,201 apps in APPSET. Figure 3 shows the distribution of the default values of PICO SDK privacy APIs.
The privacy-invading default values are not evenly distributed among the privacy APIs for different laws, i.e., the GDPR APIs of eight SDKs, the CCPA APIs of nine SDKs, and the COPPA APIs of 19 SDKs. A potential reason for this disparity could be the varied responsibility models adopted by the PICO SDKs, caused by the differing scopes of applicability across different laws. For instance, GDPR, with its broader reach, generally applies to any apps serving individuals within the EU and those based in the EU, while COPPA specifically targets apps serving children under the age of 13 in the US. As a result, many PICO SDKs have largely considered GDPR compliance as a baseline requirement, without privacy-invading default values. However, regarding COPPA, SDKs have generally adopted a model where it is the responsibility of app developers to inform SDKs of COPPA's applicability (while, without being informed, the SDK assumes that users are not children and collects their data by default, as described by Unity Ads [19]).
Another observation is that PICO SDK developers often do not disclose the default configuration of the privacy APIs, which goes against the best practices of software configuration [74]. Even more concerning, some SDKs are potentially concealing their default configurations from public scrutiny. An example is the Meta Audience Network SDK. The SDK has a "limited data use" (LDU) feature for achieving privacy compliance with US state laws including CCPA. By default, the SDK uses new String[0] parameter on its setDataProcessingOptions to completely disable LDU. The way that this default parameter is specified is suspicious. Upon initialization, the SDK sets the parameter by reading the DATA_PROCESSING_OPTIONS_KEY key from the shared preference. Since LDU has never been configured before, the shared preference will return a null, which is then replaced by the SDK with a new String[0] value. Interestingly, the above key to access shared preference is obfuscated with a byte-wise XOR encoding by Facebook (in com.facebook.ads.redexgen.X), while all the other keys are left intact. This appears uncommon since the only benefit of obfuscating this key, as far as we can imagine, is to make external reviews more challenging to identify the default LDU enablement status.
We want to note that the discovered privacy-invading default values may not fully represent all PICO SDKs. In Figure 3, we label many APIs as "Unknown, SDK-Dependent" ( ) because the extracted API parameter does not match any value recorded in the PICO METADB. For example, Vungle's COPPA API has a default COPPA_NOTSET parameter, which does not match either of the recorded values, i.e., COPPA_ENABLED and COPPA_DISABLED. While such default values could also invade privacy, we did not report them as PICO risks related to PVP #4.
Ambiguous consent configuration (PVP #5). Sixteen PICO SDKs have an ambiguous consent configuration in their privacy APIs, which, even if well configured, fails to stop the collection of user personal data. These PICO SDKs lead to unexpected and bypassing privacy exposure to 14,869 apps. One prominent piece of personal data being collected is the advertising ID, a unique ID for advertising. We believe this issue is mainly caused by the lack of standards in implementing privacy APIs. Even though PICO SDKs provide seemingly similar privacy APIs (e.g., for collecting user consent), they don't agree on the same privacy assurance.
Ineffective configuration (PVP #6). We identified that eight PICO SDKs have at least one privacy API that can become ineffective due to the presence of other program paths that affect its execution. These PICO SDKs are integrated by 2,097 apps in APPSET. Interestingly, all the eight PICO SDKs, such as MobileFuse and Appodeal, share IPSVs between their GDPR and CCPA APIs. As a result, the GDPR configuration of these PICO SDK is affected, e.g., overridden, by their CCPA configurations. The reason for this lies in the fact that, similar to MyTarget, these PICO SDKs have CCPA APIs intended to offer essentially the same privacy guarantees as their GDPR APIs, i.e., obtaining user consent for data collection. We believe this is a result of a common misinterpretation of CCPA. In reality, CCPA relies on implied user consent for data collection while requiring options that allow users to opt-out of selling and sharing personal data. Hence, obtaining user consent may not lead to full CCPA compliance, as it does not fulfill the fundamental CCPA requirement regarding opting out of selling and sharing with third parties directly.
Uneven privacy support (PVP #7). The support of privacy laws across PICO SDKs varies. For example, only 27 (41.5%) of the 65 PICO SDKs provide privacy APIs for all three privacy laws. Particularly, 14 out of the 22 PICO wrappers offer APIs for all these laws. However, most of them (11) encapsulate at least one PICO SDK that does not provide APIs for all these laws, resulting in uneven privacy support.
Broken privacy delegation (PVP #8). Even with privacy APIs to the same privacy law, 12 wrappers (54.5% out of 22) fail to propagate the privacy configurations into at least one encapsulated PICO SDK, resulting in inconsistent privacy configurations in 232 apps. Exploring the documents and design of PICO SDKs allows us to make several reasonable guesses. First, the privacy APIs of PICO SDKs often appear in the optional "advanced settings" documents. PICO wrappers are not required to configure privacy exposure of PICO SDK with these APIs before they can access the main functionality of PICO SDKs. As a result, many PICO wrappers may refrain from calling the APIs unless it becomes mandatory. Second, even in the case that PICO wrappers prioritize privacy propagation, their ability to accurately propagate configurations is often challenged by the opacity of other SDKs, and the mismatching semantics of privacy APIs. For instance, Facebook's CCPA API, AdSettings.setDataProcessingOptions, implements the "Limited Data Use" mode to ensure compliance with state laws such as CCPA, for individuals in California, Colorado, or Connecticut. Unfortunately, this may not be immediately evident to PICO wrappers, which commonly interpret CCPA as "do not sell/share" or "opt-out of selling/sharing". Therefore, as we suggest in Section 7, adopting standard privacy APIs (e.g., the IAB compliance framework [11]) may help minimize the confusion caused by the custom privacy API implementations of various SDKs.
Conditional propagation (PVP #9). Furthermore, there can be an issue even if there is a path to propagate privacy configurations. For example, three PICO wrappers, i.e., Appodeal, BidMachine, and Admost, have additional conditions such as checking IP geolocation when propagating privacy configurations to wrapped PICO SDKs. These conditions affect the propagation of privacy configurations to at least 18 PICO SDKs.
In Figure 4, we illustrate the PICO risks that occur between a series of PICO wrappers (left) and PICO SDKs (right).
Disclosure. We reported all the PICO SDKs to related vendors including SDK and app developers, which were acknowledged by some vendors recently. For example, Vungle confirmed and fixed the issues in its latest version 7.3.1; five other SDK vendors appreciated the issues we reported and are investigating them. We will update the vendors' acknowledgement at [17].
Suggestions to stakeholders. Achieving privacy compliance requires collective efforts from multiple involved stakeholders. • SDK developers. Many PICO risks, particularly those associated with PICO wrappers, result from ad hoc, non-standard, and varying implementations of privacy APIs across various SDKs. Therefore, SDK developers may consider adopting standard APIs, such as the IAB framework [11], to offer uniform and interoperable privacy API implementations. Fully integrating the standard APIs is challenging and can take a long time because it requires significant changes to both the client-side and server-side code of PICO SDKs, e.g., to fully support IAB signals. As a short-term and interim solution, without overhauling the SDKs' existing code, SDK developers may consider developing adapters on the client-side to parse, propagate, and map standard APIs to the current offerings of privacy APIs. Furthermore, missing configuration is the most common risk in the use of PICO SDKs. To mitigate this risk, it may be reasonable for SDK developers to implement a gating mechanism or debugger (as done by AppLovin) that verifies the presence of privacy APIs for each PICO SDK integration. • App developers. Apps often incorporate many SDKs, and the privacy configurations of these SDKs can be erroneous and inconsistent. However, in the regular app development lifecycle, there are no review or testing procedures specifically designed to address such errors and inconsistencies. Therefore, app developers may consider implementing dedicated reviews or test cases before delivering their apps to the general public.
• Privacy law enforcement agencies. The current PICO SDK ecosystem has generally adopted a confusing responsibility model, where PICO SDKs act as data controllers benefiting from the processing of user-sensitive data, while only the apps (rather than PICO SDKs) are exposed to pursuits by law enforcement agencies [53][54][55]. Hence, PICO SDK providers have little motivation to support materialized compliance, such as incorporating privacy-compliant defaults for COPPA. To address this issue, a more effective approach for law enforcement agencies could be to directly perform compliance checks on the implementation of popular PICO SDKs, which can also reduce compliance risks for any apps using these SDKs.
Limitation Discussion. We developed common-sense principles and PVPs to model expected practices for using and implementing PICO SDKs. Though they cover various types of privacy risks, they are not exhaustive. Other relevant principles and PVPs may exist, which we will explore in future studies. Our PICOSCAN framework primarily relies on static app/SDK analysis, supplemented with dynamic analysis for some PVPs. Due to inherent limitations in processing dynamically loaded or triggered features, the framework's results may not fully reflect app/SDK behavior. This study primarily examines client-side privacy configurations and their implications for privacy. However, apps and SDKs may also use server-side privacy configurations, potentially conflicting with or overriding client-side settings. Analysis of server-side configurations and their interactions with client-side configurations is left for future research.
In addition, this study reported privacy risks that result in technical violations of several privacy principles. However, it is possible that these risks do not actually violate privacy laws. For example, when assessing risks for the CCPA, we assume that all apps available in California are subject to its provisions. In fact, the applicability of the CCPA is contingent upon threshold requirements such as gross annual revenue or the number of California residents whose data are processed. It is likely that the apps we reported on do not meet these requirements, and therefore their privacy risks regarding the CCPA may not constitute actual violations. A similar situation may arise with privacy risks related to the GDPR, especially in cases where the risks manifest in apps that do not use consent as their legal basis. Analyzing these non-technical factors and determining actual violations to privacy laws require the involvement of legal experts and additional non-technical information, which we consider outside the scope of this study. Nevertheless, we firmly believe that identifying privacy risks is highly valuable, as it helps narrow down the analysis of actual violations. Our PICOSCAN implementation identifies PICO SDKs and their privacy APIs by matching the package names and API signatures, respectively, as recorded in the PICO METADB. As a result, the implementation is not obfuscation-resistant and cannot cover apps that obfuscate the PICO SDK code, and hence the measurement results in § 6 represent only a lower bound of the actual PICO risks.
Over the past years, a series of prior studies have evaluated the compliance of the privacy policies of mobile apps, and reported incomplete, vague, incorrect, or misleading privacy policies regarding data collection, sharing, and data usage purposes [35,36,43,66,81,82,85]. Similar work also studied the privacy policies of third-party SDKs [87]. Some other studies analyzed the issues in app consent notices, and reported the data collection and third-party tracking that either without prior consent notices [42, 50, 61-63, 68, 69], or with notices that do not meet the freely given, specific, informed and unambiguous requirements of GDPR [41,61,62,64,69,78].
Two prior studies [51,71] are related to our study since they also discussed the privacy configuration issues of third-party SDKs. Specifically, Reyes et al. [71] evaluated the COPPA options by analyzing the traffic generated by child-directed apps and reported that many of the apps fail to set up these options although their users are children. This study, relying on traffic analysis, was unable to attribute the traffic to the app code and identify the actual causes in the code that violate privacy principles, as has been done in our study. Du et al. [51] evaluated inconsistencies between user withdrawal choices and thirdparty library data collection. This privacy issue can stem from apps failing to configure SDKs for users' data withdrawal (part of PVP #2) or from SDKs failing to fulfill their data withdrawal promises by stopping data collection (part of PVP #5). However, PVP #2 and #5 in our study are more general, covering more privacy rights beyond withdrawal (or opt-out), such as user consent and child privacy rights. Particularly, compared to both related studies, we comprehensively studied the PICO SDK ecosystem and developed several privacy principles that enable the analysis of a broad range of privacy risks. These risks are represented by nine distinct violation patterns (PVP #1-#9) and cover different roles in the ecosystem, including apps, PICO SDKs, and PICO wrappers. Many of these risks arise from the interactions between these components and have not been systematically studied before.
In this paper, we conduct the first comprehensive measurement study on the privacy compliance risks of privacy-configurable SDKs (PICO SDKs) and the apps using them. We built a database of 65 popular PICO SDKs and analyzed over 48,000 Google Play apps to assess their adherence to privacy principles, covering both the app developer's code and the SDKs' code. This analysis is facilitated by our development of a compliance risk analysis framework, PICOSCAN. Our findings unveil significant privacy risks associated with both app developers and third-party SDKs, highlighting systemic issues that underscore the need for improved privacy practices and configurations in the PICO SDKs' ecosystem.
The app is subject to all three privacy laws. It is available to users in the EU and CA. It also collects users' date of birth and permits children to use it.