More Like this

1. Joint Estimation of the Arrival Rate and Customer Taste Coefficients From Censored Transactional Data

   https://doi.org/10.1177/10591478251327753  

   Wang, X; Xu, W (March 2025, Production and Operations Management)

   This work aims to jointly estimate the arrival rate of customers to a market and the nested logit model that forecasts hierarchical customer choices from an assortment of products. The estimation is based on censored transactional data, where lost sales are not recorded. The goal is to determine the arrival rate, customer taste coefficients, and nest dissimilarity parameters that maximize the likelihood of the observed data. The problem is formulated as a maximum likelihood estimation model that addresses two prevailing challenges in the existing literature: Estimating demand fromdata with unobservable lost salesand capturingcustomer taste heterogeneity arising from hierarchical choices. However, the model is intractable to solve or analyze due to the nonconcavity of the likelihood function in both taste coefficients and dissimilarity parameters. We characterize conditions under which the model parameters are identifiable. Our results reveal that the parameter identification is influenced by thediversity of products and nests. We also develop a sequential minorization-maximization algorithm to solve the problem, by which the problem boils down to solving a series of convex optimization models with simple structures. Then, we show the convergence of the algorithm by leveraging the structural properties of these models. We evaluate the performance of the algorithm by comparing it with widely used benchmarks, using both synthetic and real data. Our findings show that the algorithm consistently outperforms the benchmarks in maximizing in-sample likelihood and ranks among the top two in out-of-sample prediction accuracy. Moreover, our algorithm is particularly effective in estimating nested logit models with low dissimilarity parameters, yielding higher profitability compared to the benchmarks. 

   more » « less
2. Towards Federated Bayesian Network Structure Learning with Continuous Optimization

   Ng, Ignavier; Zhang, Kun (March 2022, Proceedings of Machine Learning Research)

   Camps-Valls, Gustau; Ruiz, Francisco J.; Valera, Isabel (Ed.)

   Traditionally, Bayesian network structure learning is often carried out at a central site, in which all data is gathered. However, in practice, data may be distributed across different parties (e.g., companies, devices) who intend to collectively learn a Bayesian network, but are not willing to disclose information related to their data owing to privacy or security concerns. In this work, we present a federated learning approach to estimate the structure of Bayesian network from data that is horizontally partitioned across different parties. We develop a distributed structure learning method based on continuous optimization, using the alternating direction method of multipliers (ADMM), such that only the model parameters have to be exchanged during the optimization process. We demonstrate the flexibility of our approach by adopting it for both linear and nonlinear cases. Experimental results on synthetic and real datasets show that it achieves an improved performance over the other methods, especially when there is a relatively large number of clients and each has a limited sample size. 

   more » « less
3. Model-Based Clustering of High-Dimensional Longitudinal Data via Regularization

   https://doi.org/10.1111/biom.13672  

   Yang, Luoying; Wu, Tong_Tong (April 2022, Biometrics)

   Abstract We propose a model-based clustering method for high-dimensional longitudinal data via regularization in this paper. This study was motivated by the Trial of Activity in Adolescent Girls (TAAG), which aimed to examine multilevel factors related to the change of physical activity by following up a cohort of 783 girls over 10 years from adolescence to early adulthood. Our goal is to identify the intrinsic grouping of subjects with similar patterns of physical activity trajectories and the most relevant predictors within each group. The previous analyses conducted clustering and variable selection in two steps, while our new method can perform the tasks simultaneously. Within each cluster, a linear mixed-effects model (LMM) is fitted with a doubly penalized likelihood to induce sparsity for parameter estimation and effect selection. The large-sample joint properties are established, allowing the dimensions of both fixed and random effects to increase at an exponential rate of the sample size, with a general class of penalty functions. Assuming subjects are drawn from a Gaussian mixture distribution, model effects and cluster labels are estimated via a coordinate descent algorithm nested inside the Expectation-Maximization (EM) algorithm. Bayesian Information Criterion (BIC) is used to determine the optimal number of clusters and the values of tuning parameters. Our numerical studies show that the new method has satisfactory performance and is able to accommodate complex data with multilevel and/or longitudinal effects. 

   more » « less
4. Urban Flood Modeling: Uncertainty Quantification and Physics‐Informed Gaussian Processes Regression Forecasting

   https://doi.org/10.1029/2022WR033939  

   Kohanpur, Amir H.; Saksena, Siddharth; Dey, Sayan; Johnson, J. Michael; Riasi, M. Sadegh; Yeghiazarian, Lilit; Tartakovsky, Alexandre M. (March 2023, Water Resources Research)

   Abstract Estimating uncertainty in flood model predictions is important for many applications, including risk assessment and flood forecasting. We focus on uncertainty in physics‐based urban flooding models. We consider the effects of the model's complexity and uncertainty in key input parameters. The effect of rainfall intensity on the uncertainty in water depth predictions is also studied. As a test study, we choose the Interconnected Channel and Pond Routing (ICPR) model of a part of the city of Minneapolis. The uncertainty in the ICPR model's predictions of the floodwater depth is quantified in terms of the ensemble variance using the multilevel Monte Carlo (MC) simulation method. Our results show that uncertainties in the studied domain are highly localized. Model simplifications, such as disregarding the groundwater flow, lead to overly confident predictions, that is, predictions that are both less accurate and uncertain than those of the more complex model. We find that for the same number of uncertain parameters, increasing the model resolution reduces uncertainty in the model predictions (and increases the MC method's computational cost). We employ the multilevel MC method to reduce the cost of estimating uncertainty in a high‐resolution ICPR model. Finally, we use the ensemble estimates of the mean and covariance of the flood depth for real‐time flood depth forecasting using the physics‐informed Gaussian process regression method. We show that even with few measurements, the proposed framework results in a more accurate forecast than that provided by the mean prediction of the ICPR model. 

   more » « less
5. Model Selection for Sparse Microbial Network Inference using Variational Approximation

   Yooseph, Shibu (March 2025, Springer Verlag)

   Microbial communities are often composed of taxa from different taxonomic groups. The associations among the constituent members in a microbial community play an important role in determining the functional characteristics of the community, and these associations can be modeled using an edge weighted graph (microbial network). A microbial network is typically inferred from a sample–taxa matrix that is obtained by sequencing multiple biological samples and identifying the taxa abundance in each sample. Motivated by microbiome studies that involve a large number of samples collected across a range of study parameters, here we consider the computational problem of identifying the number of microbial networks underlying the observed sample-taxa abundance matrix. Specifically, we consider the problem of determing the number of sparse microbial networks in this setting. We use a mixture model framework to address this problem, and present formulations to model both count data and proportion data. We propose several variational approximation based algorithms that allow the incorporation of the sparsity constraint while estimating the number of components in the mixture model. We evaluate these algorithms on a large number of simulated datasets generated using a collection of different graph structures (band, hub, cluster, random, and scale-free). 

   more » « less