More Like this

1. The Illusion of State in State-Space Models

   Merrill, William; Petty, Jackson; Sabharwal, Ashish (July 2024, International Conference on Machine Learning (ICML) 2024)

   State-space models (SSMs) have emerged as a potential alternative architecture for building large language models (LLMs) compared to the previously ubiquitous transformer architecture. One theoretical weakness of transformers is that they cannot express certain kinds of sequential computation and state tracking (Merrill & Sabharwal, 2023), which SSMs are explicitly designed to address via their close architectural similarity to recurrent neural networks (RNNs). But do SSMs truly have an advantage (over transformers) in expressive power for state tracking? Surprisingly, the answer is no. Our analysis reveals that the expressive power of SSMs is limited very similarly to transformers: SSMs cannot express computation outside the complexity class 𝖳𝖢0. In particular, this means they cannot solve simple state-tracking problems like permutation composition. It follows that SSMs are provably unable to accurately track chess moves with certain notation, evaluate code, or track entities in a long narrative. To supplement our formal analysis, we report experiments showing that Mamba-style SSMs indeed struggle with state tracking. Thus, despite its recurrent formulation, the "state" in an SSM is an illusion: SSMs have similar expressiveness limitations to non-recurrent models like transformers, which may fundamentally limit their ability to solve real-world state-tracking problems. 

   more » « less
2. Steady-State Convergence of Discrete Time State-Space Modeling with State-Dependent Switching

   https://doi.org/10.1109/COMPEL49091.2020.9265734  

   Baxter, Jared A.; Costinett, Daniel J. (November 2020, 2020 IEEE 21st Workshop on Control and Modeling for Power Electronics (COMPEL))

   null (Ed.)

   Steady-state modeling plays an important role in the design of advanced power converters. Typically, steady-state modeling is completed by time-stepping simulators, which may be slow to converge to steady-state, or by dedicated analysis, which is time-consuming to develop across multiple topologies. Discrete time state-space modeling is a uniform approach to rapidly simulate arbitrary power converter designs. However, the approach requires modification to capture state-dependent switching, such as diode switching or current programmed modulation. This work provides a framework to identify and correct state-dependent switching within discrete time state-space modeling and shows the utility of the proposed method within the power converter design process. 

   more » « less
3. The state of the aerosol: Musings on mixing state

   https://doi.org/10.1080/02786826.2024.2346220  

   Riemer, Nicole; West, Matthew (July 2024, Aerosol Science and Technology)
4. The Early Impact of the Affordable Care Act, State by State

   https://doi.org/10.1353/eca.2014.0016  

   Kowalski, Amanda E. (January 2014, Brookings Papers on Economic Activity)
5. Routine Climate Monitoring in the State of Hawai‘i: Establishment of State Climate Divisions

   https://doi.org/10.1175/BAMS-D-23-0236.1  

   Luo, Xiao; Frazier, Abby G; Diaz, Henry F; Longman, Ryan; Giambelluca, Thomas W (June 2024, Bulletin of the American Meteorological Society)

   Abstract The Hawaiian Islands have some of the most spatially diverse rainfall patterns on Earth, affected by prevailing trade winds, midlatitude disturbances, tropical cyclones, and complex island topography. However, it is the only state in the United States that does not have assigned climate divisions (boundaries defining climatically homogeneous areas), which excludes it from many national climate analyses. This study establishes, for the first time, official climate divisions for the state of Hawai‘i using cluster analysis applied to monthly gridded rainfall data from 1990 to 2019. Twelve climate divisions have been identified: two divisions were found each for the islands of Kaua‘i (Leeward Kaua‘i and Windward Kaua‘i), O‘ahu (Waianae and Ko‘olau), and Maui County (Leeward Maui Nui and Windward Maui Nui), and six divisions were identified for Hawai‘i Island (Leeward Kohala, Windward Kohala, Kona, Hawai‘i Mauka, Ka‘u, and Hilo). The climate divisions were validated using a statewide area-weighted division-average rainfall index which successfully captured the annual cycle and interannual rainfall variations in the statewide average rainfall series. Distinct rainfall seasonality features and interannual/decadal variability are found among the different divisions; Leeward Maui Nui, Leeward Kaua‘i, Kona, and Hawai‘i Mauka displayed the most significant rainfall seasonality. The western Hawai‘i Island divisions show the most significant long-term decreasing trends in annual rainfall during the past 100 years (ranging from −2.5% to −5.0% per decade). With these climate divisions now available, Hawai‘i will have access to numerous operational climate analyses that will greatly improve climatic research, monitoring, education, and outreach, as well as forecasting applications. Significance StatementThe Hawaiian Islands have some of the most spatially diverse climate patterns on Earth, but it is the only state in the United States that does not have assigned climate divisions, which excludes it from many national climate analyses. This paper establishes official climate divisions for the state of Hawai‘i, filling an incredibly important gap in the National Oceanic and Atmospheric Administration (NOAA)’s national coverage, moving toward better data equity and coverage outside the contiguous United States. Distinct rainfall seasonality features and interannual/decadal variability are revealed and compared among the different divisions. With these climate divisions now available, Hawai‘i will have access to numerous operational climate analyses that will greatly improve climatic research, monitoring, education, and outreach, as well as forecasting applications. 

   more » « less