Summary

Sniffy. For teaching Operant Conditioning. When given food, if it’s near to it, it goes to it. When does the same actions in the same place and gets food again – or it doesn’t – learns an internal model of how to get food.

Notes

https://youtu.be/jdQZdgD2afY

  • Model designed to teach operant conditioning – shaping and partial reinforcement – to psychology students. Created mainly for two reasons: financial costs and ethical challenges of using real rats in class.
  • Operant conditioning: train the rat to press the bar for food. In contrary to real rats, which must be deprived of food for 24h before the experiment, Sniffy is always hungry.
  • Primary/secondary reinforcers, shaping, conditioning, extinction behaviors. CRF (Continuous reinforcement), PRF (partial reinforcement).
  • Written in C. Animation was created out of videos of real rats, with about 15 different actions. Actions are played randomly. It is dependent on where the rat is located.
  • Rat has an internal model:
    • Guess_Responses: true if he believes is in a ratio schedule; false for interval schedule.
    • Guess_value: size of schedule.
    • Guess_fixed: it is behaving as if the schedule is fixed.
    • theory_valid: has evidence about the correctness of the schedule model.
    • The position in which he was when received food and the time between going to the position and receiving it.
  • Belief is proportional to reinforcement frequency.
  • In its conclusion, proposes to use Artificial Neural Networks.

Thoughts

  • Actions are random, not constructed based on the behavior of real rats.
  • Parameters of the simulation come with default values but can be overwritten by students. Again, it is interesting to construct better default values based on real rats.
  • Technology advances in the past 25 years make possible to use Artificial Neural Networks more easily.