Herd formation is one of the most obvious examples of emergent spatial structure in animal populations. A number of recent theory papers have addressed how behavior contributes to the emergence of animal herds. A number of methods have been used in these papers. The simplest approach is to assign movement rules to individuals without internal states and then compare the emergent spatial structures to observations of real aggregations. For example, Lee 2006 uses molecular dynamics to create snapshots of aggregations comparable to herds. More complex models may assign movement rules that depend on both internal and external states. Rands 2004 use a rule-of-thumb agent model to study emergent patterns of social foragers under predation. In other cases, models with assigned behaviors can be extended to allow behaviors to evolve. To test how selection affects aggregation behaviors, Reluga 2005 uses an agent-based simulation with a simple evolution rule to show that realistic aggregations can be stable under predation pressure. Wood 2007 combined foraging and predation and obtained more evidence that trade-offs between foraging benefits and predation risk can drive the evolution of herd behaviors.
Surprisingly, the game-theoretic foundations of aggregation behavior have not received much attention (perhaps because Hamilton's informal arguments were convincing). While thought experiments and simulation experiments have shown predation can lead to the evolution of herds, few if any mathematically rigorous results seem to have been obtained so far. Rigorous game-theoretic results would be a great addition to selfish-herd theory.
One often-invoked explanation is the Hamilton's selfish-herd hypothesis. Hamilton originally formulated his hypothesis in terms of frogs living on the circular edge of a pond. Suppose we have \(n\) frogs, and each individual \(i\) has a position \(x_i(t) \in [0,2 \pi)\) on the pond's edge at time \(t\) (for convenience, we use polar coordinates). At any time \(t\), a frog may choose to move around the circle in one direction or the other. The velocity chose by individual \(i\) at time \(t\) is \(\phi_i(t) \in [-1,1]\). The dynamics are governed by \[ \dot{x}_i = \phi_i(t), \quad \forall i \in 1 \ldots n \] At some time \(t_f > 0\), a snake emerges from the water and eats one of the frogs near to where it emerges. Thus, there a risk of death that depends on the positions of the frogs. If each frog wishes to minimize it's risk of death, how should the frogs behave?
This is clearly a game-theory problem. Given initial conditions, can we calculate a Nash equilibrium strategy-set for the movement of the frogs? To determine the Nash equilibria, we first have to specify payoffs for individual behaviors in terms of a risk function \(R_i\) for individual \(i\) that depends on the positions of other individuals at the final time \(t_f\). In the standard thought experiment, the risk of death \(R_i\) is the probability that the predator's position pops up closer to individual \(i\) than to any other individual. If individual \(i\) is between individual \(i-1\) and individual \(i+1\), then \[ R_i = \frac{|x_{i+1}(t_f) - x_{i-1}(t_f)|}{4 \pi}. \] This risk function leads to a singular model in the sense that it is independent of individual \(i\)'s position. Individual \(i\) can not change it's risk through movement. This is a subtle but important issue in the mathematical analysis. Small perturbations to the shape of \(R_i\) can lead to very different equilibria.
Thus, we have reached a bit of a paradox. Clearly, herd formation is evolutionarily stable, given the frequency with which we find it in nature. However, in theory, Hamilton's selfish-herd hypothesis -- the most frequently invoked explanation, says individuals can not change their risk of predation by movement!
A couple of caveates are in order at this point. For one, Hamilton used a discrete-time model with sequential updating and unbounded jumps in his original explanation of selfish herds, which avoids the problem we have revealed. However, in this sense, our model is more realistic than his -- animals can not teleport, but rather must move continuously through space from one point to another! A second related caveate is that in most cases, herd behavior is occurring in 2 or 3 dimensions, rather than the 1 dimension of Hamilton's thought experiment. In 2 and 3 dimensions, the size of the domain of danger DOES change according to the individual's position relative to it's neighbors. However, several simple calculations I have done suggest that moving towards a neighbor does NOT reduce one's domain of danger.
It seems that something else is going on, and some further mathematical studies are needed to justify the selfish herd hypothesis from first principles.