Title: An Analysis of Reinforement Learning Dynamics with Multiple Agents

The Q-learning reinforcement-learning algorithm is known to converge to optimal behavior in the limit in single-agent environments given sufficient exploration. The same algorithm has been applied, with some success, in multiagent environments, where traditional analysis techniques break down. Using dynamical systems methods, we derived and studied an idealization of Q-learning in 2-player 2-action repeated general-sum games. We provide a complete catalog of the convergence behavior of the epsilon greedy Q-learning algorithm. The results stand in contrast to existing results for policy-search methods. Of particular interest is the chaotic super-Nash non-convergence behavior of this algorithm in the Prisoner's Dilemma.

Title: Shrinkage estimators for out-of-sample prediction with multivariate normal data

Shrinkage estimators in statistics have a long and illustrious history. Beginning with Stein's paradox and the James-Stein estimator, it has been repeatedly observed that by "shrinking" the usual estimator for some given parameter, one can obtain better results, in terms of quadratic loss. These gains may be especially large when dealing with a high dimensional parameter or a low signal-to-noise ratio, and this observation brings heightened relevance to shrinkage estimators, given the relatively recent explosion in the availability of high-dimensional data sets. In this talk, I'll discuss the application of shrinkage estimators -- in particular, James-Stein type and ridge regression estimators -- to the out-of-sample prediction problem. Within the framework of a multivariate normal model, I'll compare out-of-sample prediction properties of several shrinkage and "usual" estimators. Previously, comparison of these estimators has been a highly technical endeavor. However, within the multivariate normal framework, such comparisons are transparent and explicit formulae for the estimators' predictive risk may be obtained. To further motivate this study, I'll discuss its relevance to the control variates problem in Monte Carlo methods and a portfolio selection problem in mathematical finance.

Bio: Lee Dicker is an assistant professor in the Department of Statistics and Biostatistics at Rutgers University. He earned his PhD in Biostatistics from Harvard University in 2010. His research interests include statistical methods for the analysis of high-dimensional datasets and proteomics.

Title: Mathematical and Computational Modeling and Analysis for Hermaphrodite Population Dynamics

This research deals with mathematical and computational modeling and analysis of hermaphrodite populations. The main objective is to study the implication of a density dependent sex allocation on the dynamic of these populations. Indeed, the sex-ratio of the hermaphrodite species depends on the sex-allocation function. It was proven for some species that in presence of sexual competition this function is density dependent and leads to complex population dynamics. Three surrounding areas of modeling were developed to study these species and to reflect their characteristics: a density deterministic approach, an Individuals Based Model approach and an Evolutionary Game Theory approach. For the deterministic approach, two structured discrete matrix models and a structured continuous model (the formulated model is a system of non linear partial differential equations) were realized and analyzed. The analysis of these models met with mathematical difficulties linked to the nature of non-linearity. However, results of stability on the equilibrium state and the asymptotic behavior of the solutions were obtained and interpreted. For the individual based approach the model allows us to take an account the variability of phenotypic (e.g. fitness) and behavioral characteristics (sexual allocation) and the interactions between individuals. This approach complements our information on the effect of sex allocation for hermaphrodite populations. In the Evolutionary Game Theory approach we will try to find the evolutionary stable strategy between different kinds of hermaphroditism depending on the different characteristics of the species. Analysis of these models proved the importance of sex allocation function form as well as survival parameters on the dynamics of hermaphroditic species. In fact, by the manipulation of the hermaphrodite life history parameters, we found the causes and the environmental conditions by which we can observe the stability, the explosion, the extinction or even a more complex dynamics like chaos.

Bio: Amira Kebir has received her PhD in 2010 in Tunisia. Her research include biological and ecological systems and modeling, also computational, quantitative, and qualitative analysis for mathematical models arising from physiological structured population dynamics. Currently, she is a postdoc at DIMACS.

Title: An Analytic Proof of Hell-Nesetril

The dichotomy conjecture of Feder and Vardi states that every Constraint Satisfaction Problem is either tractable or NP-complete. The nicest proved case is the Hell-Nesetril theorem: the H-coloring (homomorphism) problem is tractable if H is bipartite and NP-complete else. We show an analytic approach to the dichotomy conjecture. We give a transparent proof of the Hell-Nesetril theorem demonstrating this approach. Joint work with Mario Szegedy.

Bio: Gabor Kun has received his PhD in 2008 in Hungary. His research area is discrete mathematics, with applications to computer science and discrete mathematics. Currently he is a postdoc at DIMACS.

Title: Compression and Complexity of Algorithmic Problems

This talk will discuss the role of compression in complexity of algorithmic problems.

Bio: Alexei Miasnikov's research includes; Combinatorial, geometric, algorithmic and asymptotic group theory, Algorithmic and model theoretic algebra, Mathematical logic and recursion theory, Average and generic computational complexity, Cryptography, and Statistical methods in pure mathematics. He is a member of the Algebraic Cryptography Center which focuses on research in specific thrust areas including, Cryptography and Cryptanalysis, Generic Complexity, Statistical Algebra, and Assymptotic Group Theory.

Title: Stochastic foraging strategies of C. Elegans

A common model to describe the foraging movements of organisms is the random walk, in which the duration and direction of the forward movement of the organism is chosen randomly. In the absence of any knowledge of the environment, an important question is: What statistical strategy will be the most efficient to find food? Or more specifically: Which is the distribution from where the organism should choose the duration of its forward movements? Recent studies show that the notion of intermittent locomotion is important to link animal behavior to large-scale statistical properties of movement. This type of locomotion involves the existence of behavioral mechanisms resulting in sequential and episodic breaks of previous-directions: new direction not correlated with the previous one. Intermittent locomotion is tightly linked to reorientation behavior and it has been suggested that the probability distribution of time-intervals between reorientations might ultimately regulate the diffusive properties of movement at large-scales, and in turn affect the search efficiency. To answer the questions above, we studied the movement of C. elegans while foraging in a cueless environment. This is joint work with Frederic Bartumeus from Institute of Advances Studies of Blanes, Spain and with Will Ryu from Toronto University, Canada.

Bio: Liliana's research interests include the study of animal movement as a complex system. She is interested in the emergent properties that animal movement can generate both at the individual and population level. Liliana is currently completing a Computational Biology PhD from Gulbenkian Institute of Science, Portugal studying the "Ecology and evolution of foraging animal movement". She received a B.Sc (2003) and M.Sc. (2005) in Computer Science from University of Porto, Portugal where her studies focused on theoretical computer science, having worked on perfect secrecy of symmetric cryptosystems using information theory and Kolmogorov complexity, and on process-calculi and type theory to study global computing systems. For the past three years she has been a recipient of a FCT (Fundacao para a Ciencia e a Tecnologia) fellowship to conduct research abroad. The fellowship has allowed her to work with Professor Simon Levin at Princeton University for the last three years as well as with Professor Nina Fefferman at Rutgers University.

Title: Clustering time-course data

Clustering is a frequent first step in the analysis of large-scale data sets and, consequently, there are a myriad of clustering methods available. Time-courses pose specific challenges and problems as defining distances between time-courses is not straight-forward, as exemplified by data from molecular biology which frequently exhibits large variability, high dimensionality, large error rates and overlapping clusters. We will revisited mixture models, introduce some simple, but surprisingly efficient component models for time-courses, and furthermore demonstrate jointly clustering temporal and spatial pattern of gene expression during embryonal development.

Bio: Alexander Schliep received his Ph.D. in Computer Science from the University of Cologne, Germany, in collaboration with the Theoretical Biology and Biophysics Group at Los Alamos National Laboratory in 2001. From 2002 until 2009 he held a group leader position at the Max Planck Institute for Molecular Genetics in Berlin, Germany, and then moved to a position of associate professor jointly in the department of Computer Science and the BioMaPS Institute for Quantitative Biology at Rutgers. His main research themes are efficient algorithms for machine learning in bioinformatics and spatio-temporal processes in biology.