Search for: All records

With the rapid growth of large language models, big data, and malicious online attacks, it has become increasingly important to have tools for anomaly detection that can distinguish machine from human, fair from unfair, and dangerous from safe. Prior work has shown that two-distribution (specified complexity) hypothesis tests are useful tools for such tasks, aiding in detecting bias in datasets and providing artificial agents with the ability to recognize artifacts that are likely to have been designed by humans and pose a threat. However, existing work on two-distribution hypothesis tests requires exact values for the specification function, which can often be costly or impossible to compute. In this work, we prove novel finite-sample bounds that allow for two-distribution hypothesis tests with only estimates of required quantities, such as specification function values. Significantly, the resulting bounds do not require knowledge of the true distribution, distinguishing them from traditional p-values. We apply our bounds to detect student cheating on multiple-choice tests, as an example where the exact specification function is unknown. We additionally apply our results to detect representational bias in machine-learning datasets and provide artificial agents with intention perception, showing that our results are consistent with prior work despite only requiring a finite sample of the space. Finally, we discuss additional applications and provide guidance for those applying these bounds to their own work. 

The ProofLang Corpus includes 3.7M proofs (558 million words) mechanically extracted from papers that were posted on arXiv.org between 1992 and 2020. The focus of this corpus is proofs, rather than the explanatory text that surrounds them, and more specifically on the language used in such proofs. Specific mathematical content is filtered out, resulting in sentences such as Let MATH be the restriction of MATH to MATH. This dataset reflects how people prefer to write (informal) proofs, and is also amenable to statistical analyses and experiments with Natural Language Processing (NLP) techniques. 

Previous work has shown that artificial agents with the ability to discern function from structure (intention perception) in simple combinatorial machines possess a survival advantage over those that cannot. We seek to examine the strength of the relationship between structure and function in these cases. To do so, we use genetic algorithms to generate simple combinatorial machines (in this case, traps for artificial gophers). Specifically, we generate traps both with and without structure and function, and examine the correlation between trap coherence and lethality, the capacity of genetic algorithms to generate lethal and coherent traps, and the information resources necessary for genetic algorithms to create traps with specified traits. We then use the traps generated by the genetic algorithms to see if artificial agents with intention perception still possess a survival advantage over those that do not. Our findings are two-fold. First, we find that coherence (structure) is much harder to achieve than lethality (function) and that optimizing for one does not beget the other. Second, we find that agents with intention perception do not possess strong survival advantages when faced with traps generated by a genetic algorithm. 

Previous work has shown that artificial agents with the ability to discern function from structure (intention perception) in simple combinatorial machines possess a survival advantage over those that cannot. We seek to examine the strength of the relationship between structure and function in these cases. To do so, we use genetic algorithms to generate simple combinatorial machines (in this case, traps for artificial gophers). Specifically, we generate traps both with and without structure and function, and examine the correlation between trap coherence and lethality, the capacity of genetic algorithms to generate lethal and coherent traps, and the information resources necessary for genetic algorithms to create traps with specified traits. We then use the traps generated by the genetic algorithms to see if artificial agents with intention perception still possess a survival advantage over those that do not. Our findings are two-fold. First, we find that coherence (structure) is much harder to achieve than lethality (function) and that optimizing for one does not beget the other. Second, we find that agents with intention perception do not possess strong survival advantages when faced with traps generated by a genetic algorithm. 

Does structure dictate function and can function be reliably inferred from structure? Previous work has shown that an artificial agent’s ability to detect function (e.g., lethality) from structure (e.g., the coherence of traps) can confer measurable survival advantages. We explore the link between structure and function in simple combinatorial machines, using genetic algorithms to generate traps with structure (coherence) and no function (no lethality), generate traps with function and no structure, and generate traps with both structure and function. We explore the characteristics of the algorithmically generated traps, examine the genetic algorithms’ ability to produce structure, function, and their combination, and investigate what resources are needed for the genetic algorithms to reliably succeed at these tasks. We find that producing lethality (function) is easier than producing coherence (structure) and that optimizing for one does not reliably produce the other. 

Conjecturing that an agent's ability to perceive the intentions of others can increase its chances of survival, we introduce a simple game, the Hero's Dilemma, which simulates interactions between two virtual agents to investigate whether an agent's ability to detect the intentional stance of a second agent provides a measurable survival advantage. We test whether agents able to make decisions based on the perceived intention of an adversarial agent have advantages over agents without such perception, but who instead rely on a variety of different game-playing strategies. In the game, an agent must decide whether to remain hidden or attack an often more powerful agent based on the perceived intention of the other agent. We compare the survival rates of agents with and without intention perception, and find that intention perception provides significant survival advantages and is the most successful strategy in the majority of situations tested. 

We evaluate the benefits of intention perception, the ability of an agent to perceive the intentions and plans of others, in improving a software agent's survival likelihood in a simulated virtual environment. To model intention perception, we set up a multi-agent predator and prey model, where the prey agents search for food and the predator agents seek to eat the prey. We then analyze the difference in average survival rates between prey with intention perception-knowledge of which predators are targeting them-and those without. We find that intention perception provides significant survival advantages in almost all cases tested, agreeing with other recent studies investigating intention perception in adversarial situations and environmental danger assessment.