## Summary

First-order Markov chain describing a rat behavior in the elevated plus-maze. It does well qualitatively, not so great quantitatively. It has a sample size of 63 rodents.

• Suggested that since both previous models simulated the rat using sequential probabilistic processes, they could also use Markov Chains.
• Markov Chains: the behavior of the system is described with a set of discrete states. Time is also discrete. The state in a given time is dependent on the previous k states and has a specific probability. A first-order Markov Chain depends only on the current state. It is also called “homogeneous” if the transition probability does not depend on time.
• Sought to define the rodent behavior using first-order homogeneous Markov chains.
• Sample of 63 rats. 27 in the control group, 26 on anxiolytic drugs, and 10 on anxiogenic.
• Divided the maze into 11 squares. One in the center, five in each arm. What is considered as two open arms by other researchers, here is a “longer” one open arm. The same applies to the closed arm. The idea is to remove the number of states and complexity from something that’s not the goal of the research.
• Auto-transitions – go to the same state from one time unit to the other – are forbidden, since the representation intends to model the transition between squares.
• Markov Chain states: 1-5 open arms, 7-11 closed arms, and 6 center.
• The transition is considered a Bernoulli trial: only two possible outcomes. The state can change to the state to the right – probability p – or to the left – probability p-1.
• Stochastic matrix (pij):
• pij = nij / nj, where i, j = 1, 2… 11, i ≠ j
• ni = {j=1, 11} ∑ nij (i ≠ j)
• Stationary probabilities (π):
• π = πP
• Stationary probabilities (π) can be calculated in two ways:
• Solve {i=1, 11} ∑ πi = 1; or
• Pn = P × P × P × P × … × P (n times)
• They used both and considered n=30 to give a good-enough estimate. So they used the second option.
• Results show that rats under anxiolytic drug don’t have a preference for a square. Suggests that such drugs smooth out the preference difference between fear and curiosity. “as in a symmetric random walk.”
• Authors affirmed that it was already possible to describe rat’s behavior using only first-order Markov Chains. They say that preference between open and closed arms, and their transitions, seem to be simple enough for first-order. But exploratory patterns may be more complex.
• Claimed that it’s possible to tell if an unknown drug has real anxiolytic or anxiogenic effects using the models.
• Qualitatively good, quantitatively not so much.

## Thoughts

• Considering the scope of the paper, explains considerably well Markov Chains and how to interpret the results.
• Why using only first-order Markov Chains if the assumption – included in previous researches – is that the action depends on past actions, like an inertial movement? An experiment could be to try to reproduce it using more previous actions.
• Although much more understandable than last times I read about Markov Chains, it isn’t completely clear how to solve the matrices. Also, if there are other ways of representing or solving the chain problem.
• It could be interesting to also test with a more significant number of squares/states and allow auto-transitions, by measuring a transition in a time unit such as 1 second. It could improve results.
• Would it be possible to design a Markov Chain that also considers behavioral actions such as grooming?
• Based on my current readings, it is the research using the larger sample size so far: n=63. Would it have a specific reason for them to have a larger sample size, while others don’t have?