1 PHYSIOLOGY OF THE MEDIAL FRONTAL CORTEX DURING DECISION‐MAKING IN ADULT AND SENESCENT RATS by Nathan Insel __________________ A Dissertation Submitted to the Faculty of the DEPARTMENT OF PSYCHOLOGY In Partial Fulfillment of the Requirements For the Degree of DOCTOR OF PHILOSOPHY In the Graduate College THE UNIVERSITY OF ARIZONA 2010 2 THE UNIVERSITY OF ARIZONA GRADUATE COLLEGE As members of the Dissertation Committee, we certify that we have read the dissertation prepared by Nathan Insel entitled Physiology of the Medial Frontal Cortex During Decision-Making in Adult and Senescent Rats and recommend that it be accepted as fulfilling the dissertation requirement for the Degree of Doctor of Philosophy ____________________________________________ Date: October 22, 2010 Carol Barnes, Ph.D. ____________________________________________ Date: October 22, 2010 Lynn Nadel, Ph.D. ____________________________________________ Date: October 22, 2010 Katalin Gothard, M.D., Ph.D. ____________________________________________ Date: October 22, 2010 Michael Frank, Ph.D. Final approval and acceptance of this dissertation is contingent upon the candidate’s submission of the final copies of the dissertation to the Graduate College. I hereby certify that I have read this dissertation prepared under my direction and recommend that it be accepted as fulfilling the dissertation requirement. ____________________________________________ Date: October 22, 2010 Dissertation Director: Carol Barnes, Ph.D. 3 STATEMENT BY AUTHOR This dissertation has been submitted in partial fulfillment of requirements for an advanced degree at the University of Arizona and is deposited in the University Library to be made available to borrowers under rules of the Library. Brief quotations from this dissertation are allowable without special permission, provided that accurate acknowledgment of source is made. Requests for permission for extended quotation from or reproduction of this manuscript in whole or in part may be granted by the head of the major department or the Dean of the Graduate College when in his or her judgment the proposed use of the material is in the interests of scholarship. In all other instances, however, permission must be obtained from the author. SIGNED: Nathan Insel: ____________________________ 4 ACKNOWLEDGEMENTS One warm winter in Tucson I emailed two friends and long‐term lab mates, Stephen Cowen and David Euston, to consult on experiment ideas. David had pioneered the lab’s examination of the rat prefrontal cortex, and Stephen was entrenched in his own prefrontal investigations. This first meeting began what would become a weekly event, held at the coffee house or another discovered venue. We were soon joined by Kaori Takehara‐Nishiuchi, an expert on the role of rat prefrontal cortex in remembering to blink. These three scientists have helped shape this dissertation much more than the contribution already implied by my citation of their work. The behavioral task I built would not have existed without the hardware and software already laid‐down by Stephen and David, and their intellectual and scientific emphasis on accuracy over speed helped construct the ideas presented in this document. Meanwhile, Kaori has reminded me of the advantages of speed and simplicity, and has taught me how to see the data behind the staging. Other scientific contributions I owe to my mentors, including Gina Poe, Mark Bower, and Almira Vazdarjanova. Their patience and guidance have been invaluable. Michael Frank and Katalin Gothard both contributed to my scientific interests, Michael has continued to help remotely, and Kati has always been available for enthusiastic coaching. The influence of Bruce McNaughton underlies my every scientific behavior; my synapses have been trained by the expression of his scientific approach and perspective. Thanks go to Lynn Nadel for being a role model that has inspired me to be an academic and a scientist. Most importantly, thanks are overdue to my advisor Carol Barnes, the source and support for every result presented here. I would be somewhere else if she had not offered her respect and trust, and made compromises that allowed me to pursue my own ideas and interests. Help with data collection I owe to Bethany Jones, Rachel Samson, Lilian Patron, Jennifer Vega, and Zachary Wagner. Thanks especially to Lilian for taking‐over when it was most needed. Jennifer helped with histology, Jie Wang with spike‐sorting, Kim Bohne helped construct hyperdrives, and Peter Lipa I thank for intelligent consultation. Thanks to Sara Burke for rig‐sharing and help when help was needed. I am indebted to Michelle Carroll and Luann Snyder, without whom I’d have been removed from the program long ago for missing paper work and forgetting to register. Beyond scientific contributions, this dissertation required personal support from members of NSMA, students of the U.A. psychology program, La Madera, Schroon, and world‐scattered neuroscientists I see once a year. Particular mention goes to Tim Ellmore for sanity in the trenches, Christine Hagan and Neluka Leanage for invaluable help with Tanji when I was not in Toronto, and Katrin Walther for keeping my house standing when I was not in Tucson. And again, to Kaori, who makes my life better. The biggest thanks go to my family, most of all to mom and dad for providing infinity. It’s a long list, but it’s been a long journey. Thank you all for making it happen. 5 TABLE OF CONTENTS LIST OF FIGURES..........................................................................................................10 LIST OF TABLES............................................................................................................12 ABSTRACT.....................................................................................................................13 CHAPTER 1: INTRODUCTION...................................................................................15 Dissertation aims and challenges...............................................................................15 Neural circuits and decision‐making: demons vs. levers ..........................................19 Organization of the chapters......................................................................................25 CHAPTER 2: ANATOMY OF THE RODENT MEDIAL PREFONTAL CORTEX....30 Chapter aims and challenges......................................................................................30 Where is, and what is, the rat medial prefrontal cortex?..........................................30 Neurons in the mPFC and their local circuitry............................................................37 Patterns of connectivity of the medial prefrontal cortex..........................................40 Loops intrinsic to the cingulate...................................................................................42 Thalamic‐mPFC loops..................................................................................................43 Neocortical‐mPFC loops..............................................................................................46 Hippocampal formation‐medial prefrontal loops......................................................49 Basal ganglia loops......................................................................................................54 Subthalamic nucleus and cerebellum......................................................................59 Amygdala‐mPFC loops.................................................................................................60 Neuromodulatory loops..............................................................................................61 Dopamine from the ventral tegmental area...........................................................62 Acetylcholine from the basal forebrain...................................................................65 Norepinephrine from the locus coeruleus ..............................................................67 Serotonin from the dorsal Raphe nucleus...............................................................69 Non‐neurotransmitter modulators..........................................................................49 Loops with the hypothalamus and brainstem............................................................70 Endopiriform nucleus and claustrum..........................................................................73 Chapter summary........................................................................................................74 6 TABLE OF CONTENTS ‐ continued CHAPTER 3: BEHAVIORAL, COGNITIVE, AND EMOTIONAL FUNCTIONS OF THE MEDIAL PREFRONTAL CORTEX..................................76 Chapter aims and challenges....................................................................................76 General theories of prefrontal function...................................................................78 What does the dorsal mPFC do for behavior and cognition? Examining how action‐outcome expectations are represented and compared........................................................................................................84 Relationship between cognitive control and action selection.................................89 The ugly duckling prelimbic cortex: part dmPFC, part vmPFC, living in a dorsal‐lateral PFC world........................................................................94 What does the ventral mPFC do for behavior, cognition, and emotion? Examining the expectation of reward and punishment.......................................98 Forming outcome expectation from history: hypothesized role of the hippocampus in creating mPFC expectation............................................101 Training expectation by prediction error: hypothesized role of dopamine..........105 Chapter summary....................................................................................................106 CHAPTER 4: NEURON COMMUNICATION AND MODULATION IN THE MEDIAL PREFRONTAL CORTEX.........................................................108 Chapter aims and challenges...................................................................................108 A brief history of theory: network mechanisms of persistent activity and the roles of inhibition....................................................................................109 Hypothesized roles of cortical loops in persistent mPFC activity...........................114 The role of neuromodulatory loops for mPFC circuit function...............................119 Oscillations and cortical communication: focus on gamma....................................126 State changes in brain circuits during decision‐making..........................................134 Chapter summary.....................................................................................................135 CHAPTER 5: THE EFFECTS OF AGE ON THE MEDIAL PREFRONTAL CORTEX..................................................................................................................138 Chapter aims and challenges...................................................................................138 Anatomical and chemical changes of the mPFC with age.......................................141 Behavioral and cognitive changes and frontal function in the aged primate.........144 Behavioral changes and frontal function in the aged rodent.................................149 7 TABLE OF CONTENTS ‐ continued Neuron and network‐level physiological changes in old age.................................151 Chapter summary....................................................................................................152 CHAPTER 6: SUMMARY OF REVIEW, HYPOTHESES, EXPERIMENT DESIGN...................................................................................................................154 Chapter aims and challenges...................................................................................154 Summary of medial prefrontal cortex: structure, function, algorithm, and aging...............................................................................................................154 From theory to specific experimental hypotheses..................................................157 Justification of experimental choices: species, behavioral task, and probes.........161 CHAPTER 7: BEHAVIORAL PATTERNS OF YOUNG AND AGED ADULT RATS DURING DECISION‐MAKING..................................................................171 Introduction..............................................................................................................171 Aged rats are impaired at localizing auditory cues, but learn to follow visual cues at the same rates as young adults......................................172 Aged and young adult rats performed more poorly, and made slower decisions, in trials with decision conflict..................................................176 Aged and young adult rats performed more poorly when required to return to the same feeder zone.......................................................................179 Performance on a trial was not influenced by recent reward history....................181 Summary of behavioral patterns observed in young and aged adult rats..............183 CHAPTER 8: PATTERNS OF PREFRONTAL NEURON ACTIVITY BEYOND THE EXPERIMENTAL HYPOTHESES................................................186 Introduction.............................................................................................................186 Part 1: Spike timing..................................................................................................188 Neurons can be classified by the shapes of their action potential waveform and their autocorrelations.............................................................188 Neuron spike timing oscillates at gamma and theta frequencies......................192 Reciprocal exciatory‐inhibitory neuron interactions are linked to 40‐70 Hz gamma in the local field potential...................................................195 Neuron firing is phase‐locked to hippocampal theta.........................................197 8 TABLE OF CONTENTS ‐ continued Regular‐firing neurons fire more slowly in ventral regions of mPFC in task and in rest................................................................................199 Part 2: Relationship of neuron firing to measurable external variables................199 Neuron selectivity for trial phases and trials varies between mPFC regions......199 Task cues evoke network inhibition, proximity to reward increases network excitation............................................................................................204 Local oscillation activity showed similar patterns across trial phases as local inhibitory neuron activity........................................................207 Network activity is strongly driven by position and movement..........................209 Position and place‐related firing are not affected by trial number, session difficulty, or the cues presented...........................................................218 Network activity is apparently not affected by reversals, but does change over a session.......................................................................................220 Chapter summary and prelude to subsequent chapters......................................222 CHAPTER 9: NEUROPHYSIOLOGICAL CORRELATES OF BEHAVIORAL DIFFERENCES (AND SIMILARITIES) BETWEEN AGED AND YOUNG ADULT RATS..........................................................................................................225 Introduction..............................................................................................................225 Part I: Aging affects spike timing and local circuit interactions..............................226 Aged rats exhibit slower gamma oscillations than young adults........................226 Slower gamma oscillations are related to changes in excitatory‐inhibitory neuron coupling and reduced firing rates in fast‐spiking neurons..................228 Slower gamma in aged rats is correlated with slower behavioral speed, but not slower behavior.......................................................................233 Evidence for changes in local theta‐dynamics in aged rats................................236 Part 2: The preservation of neural coding with age................................................238 Selectivity for trial phase and between‐trial variables does not differ between age groups................................................................................239 Chapter summary....................................................................................................241 CHAPTER 10: INVESTIGATING THE PHYSIOLOGICAL BASIS OF THE MEDIAL FRONTALCONTRIBUTION TO DECISION‐MAKING.....................242 Introduction.............................................................................................................242 9 TABLE OF CONTENTS ‐ continued Part I: Expectation activity during the pre‐outcome period.....................................244 Neurons with differential firing prior to errors, compared with prior to rewards, included neurons coding for velocity, outcome‐expectation, and feeder value....................................................................................................244 Principle components analysis can be used to tease apart variables contributing to neuron firing changes between trials.......................249 Part II: Prospective activity during the trial‐initiation (cue‐zone) period.................254 Differential representation of cued and non‐cued, non‐entered feeders is either absent or below detection threshold...................................................255 Neuron firing during the cue‐zone period is higher on incorrect trials compared with reward trials...............................................................................257 Convergent evidence suggests that neuron activity during the cue‐period relates to goal‐directed action selection..........................................261 Chapter Summary...................................................................................................267 CHAPTER 11: DISCUSSION......................................................................................269 Dissertation overview................................................................................................269 Interpretation of performance similarities between aged and young rats on an extra‐dimensional reversal task...................................................................272 The connection between age related slowing and the speed of the gamma oscillation.............................................................................................277 Distinct network states in the cingulate between attention/decision and reward‐acquisition epochs..............................................................................283 Decisions and the medial prefrontal cortex: the importance of expectation in guiding behavior.............................................................................285 APPENDIX A: MATERIALS AND METHODS.........................................................293 APPENDIX B: HISTOLOGY........................................................................................306 REFERENCES................................................................................................................308 10 LIST OF FIGURES Figure 1.1 Illustration of reflex‐arc hierarchy and the role of feedback loops...............23 Figure 2.1 The medial‐surface of the mammalian brain.................................................36 Figure 2.2 Types of GABAergic neurons found in the rat medial prefrontal cortex.......39 Figure 2.3 Microcolumn structure of the prelimbic cortex.............................................40 Figure 2.4 Intrinsic connections of the cingulate............................................................42 Figure 2.5 Interconnection between mPFC and the MD nucleus of the thalamus........45 Figure 2.6 Dorsal versus ventral cortical connectivity in the mPFC...............................48 Figure 2.7 Interconnections between hippocampus, entorhinal cortex, and medial prefrontal cortex..............................................................................................51 Figure 2.8 Basal ganglia loops..........................................................................................59 Figure 2.9 Connections between the mPFC and the VTA................................................64 Figure 2.10 Dorsal‐ventral gradient of mPFC projections to the midbrain....................72 Figure 3.1 Schemas and cognitive control.......................................................................81 Figure 6.1 Use of different animal species for in vivo neuroscience investigation......163 Figure 6.2 The 3‐choice, 2‐cue decision task.................................................................168 Figure 7.1 Performance and trial completion time over sessions................................174 Figure 7.2 Learning rates over trial blocks following switch to the visual task............175 Figure 7.3 Proportion of trials aged and young adult rats ran to the non‐cued feeder.........................................................................................................176 Figure 7.4 Performance and trial completion times of aged and young adult across sessions..................................................................................................178 Figure 7.5 Amount of time before and after the decision point on conflict versus congruent trials.............................................................................179 Figure 7.6 Rats were less likely to return to the feeder just visited............................181 Figure 7.7 Autocorrelation of performance following correct and error trials............183 Figure 8.1 Classification of single neurons....................................................................190 Figure 8.2 Gamma and theta oscillations observed in neuron firing............................194 Figure 8.3 Local field potentials and average power spectra.......................................197 Figure 8.4 Distributions of information content...........................................................203 Figure 8.5 Task‐phase relationship with activity in different neuron groups..............206 Figure 8.6 Average gamma power changes over trial phase is generally consistent with average inhibitory neuron activity.................................................209 Figure 8.7 Proportion of neurons with activity differentiating specific, between‐trial variables across different trial phases...............................................213 Figure 8.8: Neurons selective for feeder prior to outcome and position in the cue‐zone...........................................................................................................219 Figure 8.9 Changes in behavior and neuron population activity over a session.........221 Figure 9.1 Whitened power spectra across trial phases between age groups............227 11 LIST OF FIGURES ‐ continued Figure 9.2 Relationships between LFP gamma oscillations and neuron action potentials.........................................................................................................231 Figure 9.3 Altered excitatory‐inhibitory interactions may reduce the firing frequency of fast‐spiking interneurons............................................................232 Figure 9.4. Correlations between behavioral speed and peak gamma.......................235 Figure 9.5 The degree of theta‐frequency activity in neurons was reduced in aged rats...................................................................................................238 Figure 9.6 Comparison of aged and young adult neuron selectivity for chosen feeder zone....................................................................................................240 Figure 10.1 Position of rats on the task platform during five trial phases chosen for analysis.........................................................................................................250 Figure 10.2 Principle component coefficient matrix describing neural activity Related to outcome prediction versus velocity ........................................................253 Figure 10.3 Neuron activity related to the cued but not entered arm was no higher than activity related to the non‐cued, not entered arm..........................256 Figure 10.4 Burst‐firing neurons of the dorsal anterior cingulate cortex were more active during cue‐presentation on incorrect trials..........................................258 Figure 10.5 Increased activity on incorrect trials after restricting trials according to whether the rat entered the neuron’s highest‐firing feeder arm........................263 Figure 11.1 Illustration for hypothesized slowing of processing speed resulting from slower gamma oscillations.................................................................282 Figure A.1 System of experimental control and recording...........................................300 Figure B.1 Location of histological lesions in young and aged rats..............................306 Figure B.2 Estimates of actual electrode position from tetrode turn depths and histology...................................................................................................307 12 LIST OF TABLES Table 8.1: Number of single‐neuron spike trains of each class used in the present analyses.................................................................................................191 13 ABSTRACT Convergent evidence suggests that the dorsal medial prefrontal cortex (dmPFC) makes an important contribution to goal‐directed action selection. The dmPFC is also part of a network of brain regions that becomes compromised in old age. It was hypothesized that during decision‐making, some process of comparison takes place in the dmPFC between the representation of available actions and associated values, and that this process is changed with aging. These hypotheses were tested in aged and young adult rats performing a novel 3‐choice, 2‐cue decision task. Neuron and local field potential activity revealed that the dmPFC experienced different states during decision and outcome phases of the task, with increased local inhibition and oscillatory (gamma and theta) activity during cue presentation, and increased excitatory neuron activity (among regular firing neurons) at goal zones. Although excitatory and inhibitory activity appeared anti‐correlated over phases of the decision task, cross‐correlations and the prominent gamma oscillation revealed that excitation and inhibition were highly correlated on the millisecond scale. This “micro‐scale” coupling between excitation and inhibition was altered in aged rats and the observed changes were correlated with changes in decision and movement speeds of the aged animals, suggesting a putative mechanism for age‐related behavioral slowing. With respect to decision‐making, both aged and young adult rats learned over multiple days to follow the rewarded cue in the 14 3‐choice, 2‐cue task. Support for the hypothesis that the dmPFC simultaneously represents alternative actions was not found; however, neuron activity selective for particular goal zones was observed. Interestingly, goal‐selective neural activity during the decision period was more likely to take place on error trials, particularly on high‐ performing sessions and when rats exhibited a preference for a particular feeder. A possible interpretation of these patterns is that goal representations in the dmPFC might have sometimes overruled learned habits, which are likely to be involved in following the correct cue and which are known to be supported by other brain regions. These results describe fundamental properties of network dynamics and neural coding in the dmPFC, and have important implications for the neural basis of processing speed and goal‐directed action. 15 CHAPTER 1: INTRODUCTION Dissertation aims and challenges How does the brain decide what the body should do? No experiment on the brain has yet found, hidden within the folds of neural tissue, a singular, demon‐like homunculus pulling the levers of our behaviors. This is fortunate, because such a finding would require neuroscientists to redirect their attention to explaining how the homunculus‐demon makes its decisions. If, in dissecting the brain of the homunculus, the neuroscientist were to observe a subsequent homunculus, we would begin to suspect an unsatisfying, infinite regress, and would be left without any real explanation. 1 A more inviting explanation for how the brain makes its decisions is to begin by assuming that the cerebrum houses not one demon, but many. The task of the neuroscientist then becomes not only to dissect the demons, but also to learn how they interact. We might further posit that all demons have access to the levers of our behaviors, but that they do not always agree with one‐another about which levers should be pulled. If this is the case, then it would be under such conditions of competitive, as compared to cooperative, interactions that observation of the brain may 1 A more serious philosophical transgression on this topic has been made by Gilbert Ryle (1949) 16 yield insight into the mechanisms by which the demons interact to decide on the optimal levers. The “demons” themselves may elude our eyes and microscopes, but with the appropriate tools we are capable of transiently tracking them and, through the complex interchanges of our own lever‐pulling homunculi, discern something about their nature. In the process of reverse‐engineering the nervous system computer, it is possible to lose sight of the fact that, unlike more familiar computers, it is also soft, organic tissue that changes over its life cycle. As we age, decision‐making slows. It becomes increasingly difficult to use situational factors to guide our decisions, instead we tend to engage habits and value systems that typically generalize across contexts. The physiological basis of these functional declines is not currently known, but developing this knowledge may lead to the invention of methods or tools that help us to resist the detrimental effects of old age. In other words, by extending the study of brain decision‐ making mechanisms to include the changes taking place in old‐age, even more opportunities become available to apply what is learned toward improvements of the human condition. In this dissertation, I will present a set of experiments and analyses that investigate patterns of neural communication during decision‐making, and how these are affected by aging. The target of examination is the medial prefrontal cortex (mPFC) of the rat. The mPFC has an exceptional role in goal‐directed decision making, and the 17 region is known to be particularly engaged during periods of decision conflict. Certain frontal regions also show particular sensitivity to the aging process, and its investigation therefore has the potential to reveal the nature of age‐dependent cognitive declines. But why study the rat? For a century rats have been the primary choice in non‐human neuroscience investigation, and there are multiple advantages to building on this scientific scaffolding in studying the properties of mammalian decision‐making and aging. 2 There are two overarching hypotheses that have guided the design of the present experiments: 1) the mPFC is a site where a comparison is made between the relative advantages of available actions, 2) the action‐comparison process taking place in the mPFC loses efficacy with old‐age. Examination of these hypotheses will be made across two levels of analyses: movement patterns of aged and young adult rats performing a decision task, and the firing activity of individual neurons and neuron populations during task performance. This investigation faces many challenges, not least among them is the inherent limitations of confirming or rejecting a hypothesis on the cognitive functions of a region by examining the activity of a small fraction of the region’s working parts. An appropriate analogy might be made by imagining a hypothetical ecologist seeking to confirm a hypothesis that certain insects help maintain the floral diversity of a desert ecosystem. A scientist would normally approach this question by establishing both an independent variable to manipulate, the presence of the insects in the region, and a 2 Justification for choosing rats as an experimental model is discussed in Chapter 6 18 dependent variable to measure, the variance in plant species. The hypothetical ecologist decides instead to address the scientific questions by tracking foraging behaviors of individual ants. Although valuable information might be learned from the ant movements, particularly with regards to the mechanisms by which floral diversity may be maintained, 3 the ability to directly confirm the original hypothesis is limited by the available knowledge on how ant movements are connected with ecological interactions of a region’s flora. Likewise in brain science, there are still unbridged rifts in our knowledge of how the firing of individual neurons participates in behavior and cognition. A valuable tool to link these levels of analysis will be to better understand the interacting units intermediate between neuron firing and behavior; that is, to learn what specific computations are performed by the cortical circuitry. Donald Hebb (1948) used the concept of a cell assembly to bridge the physiological and psychological domains. Since Hebb’s time, a great deal has been learned about neuron networks, and it has become necessary to further unpack the idea of assemblies to account for the dynamic computations made by forebrain networks. In the first half of this thesis I will review what is known about the circuitry of the mPFC, and the specificity of its decline in old age. This will include the anatomy of the system (the system’s material and the form it takes), the behavioral effects of its removal or inactivation (the system’s function), and a number of specific physiological 3 The topic of mechanism will be taken up in chapters 3 and 4, and the reader is referred to writings of the philosopher of science Carl Craver (Machamer et al., 2000; Craver, 2002; Craver, 2005) 19 and computational properties putatively important for building a mechanistic theory of its function (the algorithms by which the sysem functions). In the second half of the thesis, I will describe a series of experiments aimed at contributing to the field’s understanding of the physiological mechanisms of mPFC function and its changes in old age, by examining the mPFC of rats performing a decision task. The dissertation will encounter many loops. Loops are the rule of anatomical connectivity and consequent physiological computation. Loops are also the rule of the “Popperian” cycles of scientific observation, theory creation, testing, and observation involved in building our theoretical model of mPFC circuit function. But while the science, and the brain, may be organized in infinitely connecting loops, the reader’s time is linear (at least approximately), and also limited (always limited). The chapters are therefore organized in an attempt to accommodate these constraints. This work is one discussion of many on the mechanisms of animal behavior. It fits into a far larger historic and intellectual context that, before descending into the details of the mPFC and its role in decisions, is worth elaborating on. Neural circuits and decision‐making: demons vs. levers This chapter begins with the idea that multiple demons may be interacting to pull the levers of our behaviors. If true, then what qualitative distinction distinguishes 20 between the high‐level “demons” and their presumably less animate levers? We might begin testing the usefulness of the idea by equating the levers with a number of relatively isolated neuron circuits in the brainstem and spinal cord that, when stimulated, cause specific but complex behaviors (e.g., locomotion) to be initiated or halted. These circuits are more commonly called “reflex‐arcs” and “central pattern generators.” We have some understanding of how the environment controls these circuits, thanks in large part to guiding principles articulated by Sir Charles Sherrington more than one hundred years ago. In The Integrative Action of the Nervous System, Sherrington (1906) posited that the nervous system is organized into hierarchies of reflex‐arcs. At one end of a given arc, the sensory environment is transduced into neural signals. At the other end, neural signals cause changes in the body, for example by causing the contraction of muscle cells. Lower levels of the reflex‐arc hierarchy are defined by connections that are direct, or nearly direct, between sensory‐receptor neurons and neurons that terminate on effector organs. Higher levels of the hierarchy involve multiple synaptic contacts, enabling the integration of signals from multiple sensory receptors, which eventually leads to inhibition or excitation of the same effector neurons. The effector neurons (or motor neurons for cases in which they terminate onto muscle cells) are also called the “final common pathway,” because they are required for all actions, independent of where the action commands are initiated. The principle of a reflex‐arc hierarchy is fundamental to understanding how actions are decided upon by the nervous system. One author, Joaquin Fuster, whose 21 theory of prefrontal function will be addressed later, describes the nervous system as a hierarchy of reflex arcs, with the prefrontal cortex at the top (Fuster, 2008). From this perspective, there is no need to invoke the image of a homunculus demon: we are composed of a hierarchy of lever‐switches connected by AND and OR gates. On the other hand, there are many ways in which a lever‐only theory of brain decision‐making fails as an explanation. The networks of neurons in the telencephalon exhibit a number of properties that qualitatively distinguish them with respect to how information is processed. For one, there is an increased reliance of feedback loops. The term “reflex‐ arc” connotes a feed‐forward system, from sensation to action. Although most central pattern generators involve some sort of feedback, at higher‐levels of brain processing feedback loops become the rule. Feedback loops help stabilize network activity states, they allow “efference copies” of output patterns to be processed as inputs, and they enable incoming information to the system to be filtered as a result of feedback‐based prediction of that input (Sherrington, in fact, hypothesized this to be the defining function of the cerebral cortex; see also Figure 1.1). A second feature that distinguishes network behavior in higher‐order and lower‐order regions are the dynamics by which neural assemblies are created and broken. Lower‐level reflex arcs are thought to be relatively rigid, in the sense that one component of the circuit tends to always be co‐active with another component based on the synaptic weights between them. At higher levels, unique but stable assemblies of neurons can be rapidly established or broken. Readers who have not already abandoned the analogy of levers 22 and demons may find convenience in equating these dynamic assemblies, or the dynamics of their emergent formation, with the decision‐demons. 4 4 Taken to their literal extreme, these statements do not do justice to the complexity of circuits such as are found in the spinal cord, which also exhibit network state changes that can putatively be called “dynamic assemblies.” In Chapter 4, elaboration will be made on the meaning of dynamic assemblies in the neocortex, including how cortical regions acting on one‐another synchronize within the gamma oscillation. Assembly formation that takes place within environments of large‐scale networks, such as the neocortex, is likely to be qualitatively different from that which takes place in more isolated circuits, such as those found in the spinal cord. For references, Edelman (1993) discusses the theory of neuron group selection in large‐scale networks, and has written several books on how the theories can account for cognitive phenomena such as consciousness. Buzsaki (2006) also covers these topics, emphasizing the importance of power‐law organization and various frequencies of oscillations. 23 Figure 1.1 Illustration of the reflex‐arc hierarchy and the role of feedback loops. From the outside‐in this figure represents 1) the peripheral sensory and motor systems, 2) central nervous system regions involved in perception and action, and 3) the overlapping representations of environmental percepts and movement schemas in higher‐order brain regions. The “higher‐order” decision centers, represented in the center of the figure, show a cycling between action and outcome that is meant to represent the brain’s use of feedback‐loops in representing possible outcomes of available actions, without requiring the actions to be performed externally. Feedback loops in the cortex are also important for expectation‐based filtering of sensory information and network stability (not depicted here). The overlapping color squares in the center of the figure are meant to represent the increased integration of environmental, internal, and action states that might be used to generate goal‐directed 24 action schemas. Arrows depict influential interactions between systems. A very large red arrow points from this looping, central system toward the “somatic motor” box in the upper left—this arrow is meant to depict the dissertation focus: how do higher‐ order representations of goal and decisions evoke a specific action to be performed? The concepts described in the figure are similar to those described by Sherrington and Fuster (see text). From the system of reflex arcs conceived by Sherrington, we then enter a realm of assembly‐formation through systems of connected loops—the ideas central to Donald Olding Hebb’s (1948) theory of psychology. In articulating these concepts, Hebb had to distinguish them both from “switchboard” theories, positing that behavior was generated by feed‐forward sequences of AND and OR operators (here we might call them “lever‐only” theories), and from “field theories”, which postulated that mental states somehow arose from the configuration of physiological electrical activity (in which case the “demons” are not bound to neural connectivity at all!). Hebb pointed neuroscientist in the necessary direction for understanding decision‐making networks, but we now know that the functions performed by feedback‐loops are much more elaborate than could be foreseen at that time. Many of these circuits, which are capable of performing a wide range of serial and parallel mathematical transformations, meanwhile being trained by other circuits that replay and reinforce, loop through and involve the prefrontal cortex. 25 The prefrontal cortex is uniquely situated to manage vast amounts of information, 5 and injury to different areas of prefrontal cortex can cause a wide range of actions that are inappropriate to the specific context. It is therefore a region of exceptional value in investigating how the brain selects particular action sets under conditions when available information is in conflict. Moreover, many of the functional declines experienced by aged adults can be linked to changes observed in the prefrontal cortex and the systems of modulator neurotransmitters that it makes use of. Aged adults are increasingly impaired at selecting the appropriate action as the amount of information that must be integrated, both from the past and from the present, increases. Aged adults are also, in a very general sense, slower in their behaviors and decisions. Organization of chapters Since the material of neural computation is neuron connectivity, a thorough understanding of the anatomical architecture of the mPFC provides a valuable starting point to build mechanistic theories of function. Chapter 2 will review what is known about the anatomical loops of the mPFC. The review will cover what is known about 5 Here “information” does not refer to entropy, or the number of states that can be exhibited by the prefrontal cortex. The prefrontal cortex is in a position to manage large amounts of information in that environmental stimulation of the animal, and the muscular contractions of an animal, contains a large degree of redundancy. The brain reduces the redundancy in information through hierarchical levels of processing, creating an efficient, information‐rich coding scheme. The prefrontal cortex is directly connected with many regions that carry information‐dense codes. 26 how neurons within the mPFC are wired together as well as how the mPFC is connected with other regions of the brain. Specific loops that will be covered in some detail include the mPFC connection with the thalamus, the neocortex, the basal ganglia, the hippocampal formation, the amygdala, and neuromodulatory nuclei. Several additional anatomical loops will only be mentioned for lack of space and time, not necessarily lack of importance. These include the mPFC connection with the subthalamic nucleus, hypothalamic nuclei, and the claustrum. This chapter will provide the backdrop for understanding the theory of how the mPFC coordinates bodily reactions to the environment, and to the intrinsic activity of the brain. Psychological theories of mPFC function are anchored in anatomy, but also on the functional consequences of specific manipulations to the anatomy, including pharmacological effects on neurotransmitter systems. Chapter 3 will review these data within the theoretical frameworks that have been proposed for mPFC function. The field of systems neuroscience spends a large portion of its energy studying the connection between the brain and cognition/behavior, rather than examining the specific computational roles one loop or region has in relation to the other loops and regions. Chapter 4 will more closely address what is known and understood about neuron communication, computation, and modulation. The intent of this chapter will be to improve the bridge between mPFC anatomy and behavior by focusing on physiology and network computation. 27 Chapter 5 reviews the changes known to take place within the mPFC in the aging adult. This includes anatomical changes, where both synaptic and neuromodulatory factors stand out in their sensitivity to age. The relationship between behavioral and cognitive changes taking place during aging and the behavioral effects of frontal damage will also be addressed. The chapter will conclude with a brief survey of the physiological changes known to take place in the aged rat frontal cortex. Chapters 2 through 5 comprise a review of what is known about the mPFC, specifically the rodent mPFC, and how it ages. It is against this backdrop that specific questions will be addressed in the present experiments. Chapter 6 will bridge the two halves of the thesis by discussing several general hypotheses, and by justifying experimental choices, including the rat as a representative of the mammalian class, and the behavioral paradigm that is used: a freely‐moving, 3‐choice, 2‐cue decision task. An overview of the methods for the following experiments will also be presented, with details reserved for Appendix A. Chapter 7, the first results chapter, will describe the behavior of aged and young adult rats performing the 3‐choice, 2‐cue decision task. The chapter will describe how conflicting input affects decision‐making in the two age groups at various stages of value‐learning. It will also provide an overview of how the aged and young adult rats perform on the task, and how their behaviors shift following a reversal of cue‐reward contingencies. 28 Theories on the mechanisms of decision‐making and behavior rely on basic physiological observations. Chapter 8 dissects these patterns, from the patterns of action‐potential timing in single neurons, to pairs of neurons, to populations of neurons including the generation of the local field potential. Spiking patterns throughout the brain often carry information about an animal’s environment, internal state, and motor behavior; the relationships between these variables and neuron activity will be described. The observations made in Chapter 8 on neurophysiology and its relationship with behavior will be extended in Chapter 9 by their examination in aged rats, with respect to the behavioral impairments observed in aged rats on the decision‐making task. The most notable change in the aged rats, a generalized behavioral slowing, will be evaluated with respect to neurophysiological oscillations that define some forms of processing speed in the brain. Chapter 10 combines neurophysiological observations observed in young and aged rats to address specific questions on the mechanisms of decision‐making. The chapter examines principle components of neural coding to reveal prospective activity of outcomes and of entered zones. Many mPFC neurons increase firing rates prior to a rat deciding to approach a particular goal, which can be observed more strongly when rats are not following the learned rewarded cue (that is, stronger pre‐decision, zone‐ selective activity is observed on error compared with rewarded trials). This activity may 29 provide further evidence for the importance of the mPFC in goal‐directed action as compared with stimulus‐response systems involved, for example, in cue‐following. Finally, the findings of the thesis will be integrated with the introduction, and new theories will be discussed, in Chapter 11. In the words of the historic thinker Nigel Tufnel when describing a solution to the ceiling‐effect of a base‐ten system: “It goes to eleven.” 30 CHAPTER 2: ANATOMY OF THE RODENT MEDIAL PREFRONTAL CORTEX Chapter aims and challenges The goal of the present chapter is to review the connectivity patterns of the rat medial prefrontal cortex (mPFC). The behavior of networks is largely a function of the connections between the network nodes, in this case, the synaptic connectivity between neurons. Anatomical investigations have provided a generally coarse view of connectivity, but one that is critical for understanding how the mPFC integrates signals and contributes to brain function. Although knowledge of mPFC neural connectivity is highly incomplete, an exhaustive review of what is known would require a navigation device more advanced than those currently provided by Google. Details in the present outline tend to be reserved for connectivity with higher‐level brain regions (as opposed to, for example, the spinal cord, to which the mPFC is also connected), and circuits that have been associated with specific cognitive and emotional functions. Where is, and what is, the rat medial prefrontal cortex? 31 The cortex is the outermost layer of the brain. It is said that Jeff Hawkins, creator of the Palm Pilot and author of On Intelligence (2004), begins an explanation of the cortex at dinner parties by laying six business cards on top of one‐another to describe its thickness and laminar (layered) organization, then waves an unfolded dinner napkin to describe the approximate surface area of the human cortex. This layered sheet exists in all mammals, although it can be traced to before the evolutionary fork between reptiles and mammals. Reptiles have a neo‐pallium that receives thalamic input and, like the cortex, is differentiated in the fetus from the roof of the rostral end of neural‐fated tissue called the telencephalon (Northcutt, 1981). To a very rough approximation, the functional organization of the cortex can be described in dual partitions. The first partition is made by the central sulcus, an inner‐ fold that divides anterior and posterior hemispheres. Posterior to the central sulcus, generally perceptual information is processed, originating in peripheral sensory receptors and relayed by the thalamus. The cortex anterior to the central sulcus generally processes “action” information, based largely on the sensory information entering from posterior cortex and on memory and reinforcement systems, such as those found within the basal ganglia. “Actions,” in quotes, refers to the fact that feed‐ forward circuits lead not only to contraction of skeletal muscle (the somatic motor system), but also to changes in the body’s visceral system, routed through the autonomic nervous system and endocrine system. To stretch the word’s definition even further, “action” also includes the feedback systems that help control which stimuli in 32 the environment are being processed (attention) and which memories are being retrieved, which in turn facilitate the planning and execution of actions. A second dual partition distinguishes cortical control of somatic as compared to visceral systems in the body. To a rough approximation (and not always generalizing across species), somatic processing tends to be situated more dorsally (and laterally), and visceral more ventrally (and medially). Not by coincidence, this dorsal‐ventral distinction is overlaid with a similar distinction in posterior (sensory) regions of cortex: more dorsal regions tend to process information on where things are in the environment, with more ventral regions processing information about the associative identity (including value) of a stimulus. 6 There is a strong connection between the systems identifying where the body is in relationship to the environment (dorsal‐ posterior) with how the body should move within the environment (dorsal‐anterior). There is also a strong connection between the systems identifying what is in the environment (ventral‐posterior) and how the body should feel about those things (ventral‐anterior). A major task in discovering the mechanisms of decision‐making is to identify the means by which the “ventral” system of identification and value informs the “dorsal” system of space and movement. 6 Although dorsal vs. ventral sensory processing has been studied most in the visual system (Ungerleider and Mishkin, 1982), auditory computations may also be divided in this way (Kaas and Hackett, 2000), and the distinction may extend as far as the hippocampus (see also Fanselow and Dong, 2010, for somatic versus visceral distinctions of the hippocampus). 33 The region of cortex that is referred to as “prefrontal” begins at the anterior tip of the brain, and extends in the posterior direction as far as an anatomist deems still worthy of the “pre‐“ prefix. The prefrontal cortex does not include primary motor cortex (M1), adjacent to the central sulcus, and also usually omits certain areas of premotor cortex, the regions predominantly projecting to M1. Various subdivisions of prefrontal cortex can be identified based on cytoarchitectonics, patterns of connectivity with other cortical regions, and patterns of thalamic projections. Consistent with the anatomical distinctions, different cognitive and behavioral deficits can be caused by damage to these different subregions, and techniques of imaging regional activity in the brain find that areas are differentially engaged dependent on the task. The specific anatomical techniques for identifying prefrontal cortex and distinguishing between prefrontal regions might appear to be a detail beyond the interest of this thesis; in fact, this issue has become a source of controversy that still defines the field of prefrontal research in the rat (Preuss, 1995). The first detailed account of the rat frontal cortex was made by Brodmann (1908), who, based on cytoarchitechtonic grounds, came to the conclusion that rodents, along with most mammals, lacked many of the frontal regions that could be characterized in primates. Specifically, in primates the prefrontal cortex contains a granular layer (i.e., a layer IV with distinct, small cell bodies) that is absent in other species. Rose and Woolsey (1948) chose another index to characterize prefrontal cortex, the projections of the medial dorsal nucleus (MD) of the thalamus. In the early anatomical tracing experiments, MD 34 appeared to project only to granular cortex in the primate; consequentially, Rose and Woolsey inferred that the evolution of primates included the addition of a granule layer, rather than the differentiation of an altogether new region of cortex. The claims made by Rose and Woolsey were further developed by Akert (1964), who divided primate prefrontal regions into three regions according to the differential projections of the three subgroups of MD neurons: orbitofrontal cortex receiving medial, magnocellular projections; dorsolateral prefrontal cortex receiving input from the lateral, parvocellular neurons; and arcuate cortex (also dorsolateral) receiving projections from large neurons in the farthest lateral part of the nucleus. The rat medial prefrontal cortex is robustly interconnected with the MD nucleus (although, unlike in the primate, this nucleus is homogeneous in cell types; Bentivolglio et al., 1993). This provides the basis for an entire field of study that treats the rat medial prefrontal cortex as a homologue of the primate dorsolateral prefrontal cortex. The major error in this interpretation (reviewed by Preuss, 1995) is that the cingulate cortex in the primate is also robustly interconnected with MD (e.g., Giguere and Goldman Rakic, 1988). In contrast to the dorolateral prefrontal cortex of the primate, which arguably shares few similarities with the medial frontal regions of the rat, anatomical connectivity of the primate cingulate cortices is very much in agreement with the anatomy of the rat cingulate (primate: Gary et al., 1993; rat: Hoover and Vertes, 2007; Conde, 1995; Gabbot et al., 2005; Sesack et al., 1989; see also Vogt et al., 2004). Parcelations of cingulate regions in the rat, based on intrinsic connectivity (Jones et al., 2005) and descending 35 efferents (Gabbot et al., 2005), also parallel those observed in the primate (Beckmann et al., 2009). Together, the connectivity patterns describe the cingulate as a target of projections from a wide range of cortical and non‐cortical areas, and many of these afferents are directly reciprocated back by axons from the cingulate. For the extent of this document, “medial prefrontal cortex” (mPFC) will refer to the frontal regions of the rat cingulate. By the terminology of Krettek and Price (1977), these include, ventrally to dorsally, the infralimbic cortex (IL; area 25), the prelimbic cortex (PrL; area 32), the dorsal anterior cingulate cortex (dAC; area 24b), and the medial precentral region (PC; See Figure 2.1). PC has also been called the rostral forelimb region, as microstimulation can evoke forelimb movements (Neafsey and Sievert, 1982; but see also Donoghue and Wise, 1982, who find no evoked responses from electrical stimulation of the region). PC is better described as “premotor” cortex than cingulate; PC, dAC, and PrL will often be discussed independently from one another. 36 Figure 2.1 The medial‐surface of the mammalian brain. Using cytoarchitechtonic divisions, Brodmann labeled subsequently more ventral regions of the rabbit medial‐ frontal cortex 24, 32, and 25 respectively (middle) by comparison with the medial‐ frontal cortex of the human (top). Krettek and Price (1977) subsequently used the nomenclature dorsal and ventral anterior cingulate cortex (ACd, Acv), prelimbic cortex (PL), and infralimbic cortex (IL) to identify these respective regions in the rat (bottom). They also included the medial precentral region (PrCm) dorsal to the cingulate. Top 37 figures are modified from Brodmann (1909) bottom figure from Krettek and Price (1977). Neurons in the mPFC and their local circuitry Neurons throughout the cortex are organized into six or fewer layers. Since mPFC lacks a granular layer IV, neurons in the mPFC are distributed across cortical layers I‐III, V, and two subdivisions of layer VI. There are multiple types of neurons across these layers, and the connections between them are highly regular. The broadest classification of mPFC neuron types distinguishes between pyramidal neurons and local circuit (inter‐) neurons; the former being excitatory neurons releasing the neurotransmitter glutamate, the latter inhibitory, releasing gamma‐aminobutyric acid (GABA). Among pyramidal neurons, there are small, medium, and large pyramids, as well as two neuron types unique to the cingulate: extraverted and fusiform pyramidal neurons (Vogt, 1993). The extraverted and fusiform pyramids are found in layer II, with some fusiform pyramids also found in layer III. The relative functions of these different neuron classes for neural communication are undetermined. With respect to interneurons, multiple classes can be discriminated on the basis of morphology (which determine their connectivity), calcium binding proteins, neuropeptides, and ion channel genes (which determine their firing properties; Markram et al., 2004). Within the mPFC, neurons tend to fall within the following categories of morphology and associated calcium binding protein (Gabbott et al., 1997): 38 double bouquet and bipolar neurons express calretinin (CR), basket and chandelier neurons express parvalbumin (PV), and double bouquet, Martinotti, and neurogliaform neurons express calbindin D‐28k (CB). The PV‐staining basket and chandelier neurons have several distinctive properties, including that they terminate on the soma and axons of local pyramidal neurons, they are widely interconnected with pyramidal neurons, they are often connected with one‐another via electrical gap‐junction connections, and the ion channels they express make them fast‐spiking (FS) neurons with particularly narrow waveform shapes. As will be described later, these neurons likely play a large role in modulating the level of inhibition in the local circuit, and in helping to generate a 30‐80Hz oscillation known as the “gamma” rhythm (Cardin et al., 2009). The total percentage of GABAergic neurons in the prelimbic and dorsal anterior cingulate has been identified as 16.5 and 17.2% respectively (Gabbott et al., 1997). PV neurons comprise roughly 5‐6% of the total population, CR neurons around 4%, and CB neurons 3‐4%. The cell bodies of these neurons are found in all layers, although the majority can be found in layers 3 and 5 (Figure 2.2). Most GABAergic and gluatmatergic receptors are located in superficial layers of mPFC. An exception is the glutamatergic kainate receptor, which is most dense in deep layers (Palomero‐Gallagher and Zilles, 2004). 39 Figure 2.2 Types of GABAergic neurons found in the rat medial prefrontal cortex. Abbreviations refer to neurons positive for calcium‐binding proteins parvalbumin (PV), calretinin (CR) and calbindin (CB). Figure from Gabbott et al. (1997). In the prelimbic cortex in particular, connectivity between local neurons is organized in a highly systematic way. The apical dendrites of pyramidal neurons in layers V, IV, and II ascend through the layers of cortex in bundles roughly 44um apart (yielding 590 bundles within a square millimeter of cortex). On reaching layer 1, the dendrite bundles form apical tufts. Each bundle contains approximately 55 apical dendrites, or half of that if examining only the dendrites at layers III and V. Each hexagon around a bundle, operationally referred to as a “microcolumn”, contains roughly 79‐80 pyramidal neurons and 16 interneurons, many of which are PV‐staining neurons. About 9‐10 of the pyramidal neurons in each minicolumn are thought to 40 project to the MD nucleus of the thalamus (Gabbott et al., 2005). Minicolumns are not unique to the rat prelimbic cortex, and many have theorized that the minicolumn represents a fundamental unit of cortical computation or communication in the brain (Mountcastle, 1997; Innocenti and Vercelli, 2010). The issue of minicolumns as fundamental modules of cortex will be returned to later, along with evidence that neurons within a microcolumn are wired in a way that in fact decorrelates their firing (Ecker et al., 2010; Renart et al., 2010). Figure 2.3 Microcolumn structure of the prelimbic cortex. Bundles of approximately 55 apical dendrites are observed roughly 44mm apart in the prelimbic cortex (left; from Gabbott and Bacon, 1996). From these, hexagonal “microcolumns” can be inferred that each contain close to 60 pyramidal neurons and around 10 inhibitory neurons distributed through the cortical layers (right; from Gabbott et al., 2005). Patterns of connectivity of the medial prefrontal cortex 41 Several general principles help describe connectivity in the mPFC. 1) Afferents and efferents are wide‐ranging, from all areas of the brain. This means there is an impressive convergence of information to the mPFC and widespread control by the mPFC over other cortical regions. It also means robust modulation of activity and plasticity by non‐GABA and glutamate‐based neurotransmitter systems. 2) The afferents and efferents are situated in a dorsal‐ventral gradient (Neafsey et al., 1993; Heidbreder and Groenewegen, 2003; Gabbott et al., 2005; but see exceptions, described below and in Vertes, 2004). More dorsal regions tend to be connected with the sensory‐motor system, while more ventral regions tend to be connected with visceral, emotional, and modulatory systems. There is no apparent strict line that separates the dorsal from ventral, and afferents and efferents that are observed in abundance at one end of the dorsal‐ventral axis, usually appear to have at least sparse representation at the other end. 3) Because most regions projecting to mPFC also receive mPFC input, there is a high degree of looping which must have a fundamental, functional importance. The looping is perhaps most explicitly studied with respect to the loops generated by frontal cortex and the basal ganglia (Middleton and Strick, 2000), but the same concepts can be applied to virtually all connections made by the mPFC. Loops intrinsic to the cingulate 42 Generally, the mPFC has a high amount of intrinsic connectivity. Interpretation of the anatomical data requires some caution, as PrL has a dorsal and ventral subdivision that may be connected differently with the rest of mPFC. Within IL there is high connectivity, but IL is only connected with ventral PrL (Fisk and Wyss, 1999; Jones et al., 2005). The degree to which ventral and dorsal PrL are connected has not been explicitly examined. Dorsal PrL is highly connected with dorsal anterior cingulate (dAC). The most dorsal region, the medial precentral area (mPC), appears to have high intrinsic connectivity, but only minor connectivity with the rest of mPFC; connections that are present are primarily with the most dorsal portion of dAC. Intrinsic connections of the cingulate are illustrated in Figure 2.4. Figure 2.4 Intrinsic connections of the cingulate. A summary of the intra‐cingulate connections by Jones and Witter (2005). A minor addition to the original figure is the 43 placement of the red dashed line within the prelimbic cortex (PL), to emphasize the utility of distinguishing between dorsal and ventral regions of prelimbic cortex. Arrows indicate direction of projection, circles indicate origin of projection. Abbreviations: IL = infralimbic cortex, PL = prelimbic cortex, ACd = dorsal anterior cingulate cortex, ACv = ventral anterior cingulate cortex, RSd = dorsal retrosplenial cortex, RSv‐b = ventral retrosplendial cortex “b”, RSv‐a = ventral retrosplenial cortex “a”. Thalamic‐mPFC loops The connections between mPFC and thalamus are highly reciprocal (Bentivoglio et al., 1993). The nuclei of the thalamus connected with mPFC include the anterior, medial dorsal (MD), midline, anterior intraliminar, and parafascicular. The ventral anterior (VA) and ventromedial (VM) nuclei, which receive projections from the basal ganglia, as well as the intralaminar nuclei which receive basal ganglia, cerebellar and autonomic afferents, will be discussed below. The anterior‐medial (AM) nucleus, which is part of a greater loop with the temporal lobe that includes the mammilary bodies, will also be discussed below in the context of the hippocampal‐mPFC loops. The most prominent nucleus interconnected with mPFC is MD. As noted above, in the primate different cell‐types of MD project to different regions of frontal cortex. Although not observed in the rat, there is still topography in the projection patterns, with mPFC receiving primarily input from the lateral portion of the nucleus (and orbitofrontal the medial portion). Some of the same neurons which project to MD, predominantly located in layer VI (Gabbott et al., 2005), also appear to receive MD input, creating a monosynaptic, recurrent loop (Kuroda et al., 1998; Figure 2.5). Some 44 layer III neurons that project to other regions of cortex also receive MD inputs. Axons from MD primarily end in deep layer III (presumably substituting for the absent layer IV), but also terminate in layers I and, to a smaller extent, layers V and VI (Vogt et al., 1981). Many regions of the brainstem (serotonergic, adrenergic, cholinergic) also project to MD. MD is highly, topographically connected with the reticular nucleus of the thalamus, a region that receives input from the mPFC, but does not project back out to cortex. The projection from the reticular nucleus to MD is GABAergic, and likely provides a specific function within the mPFC‐MD circuit. Together, MD appears to provide an interface by which systems such as the basal ganglia (discussed below) or reticular nucleus can influence or filter the information processed in the mPFC. 45 Figure 2.5 Interconnection between mPFC and the MD nucleus of the thalamus. Excitatory neurons in the medial dorsal (MD) nucleus of the thalamus are reciprocally connected with layer VI pyramidal neurons in the mPFC. MD neurons also synapse onto inhibitory interneurons and layer III neurons that project to other regions of cortex. 46 Several other prominent inputs to the MD are shown radiating around the MD neuron; note that the illustration is not intended to suggest that the other regions project to identical MD neurons as are reciprocally connected with the mPFC, nor are connections between cortex and these regions illustrated here. Figure adapted from Kuroda et al. (1998), with the addition of non‐cortical afferents to MD also indicated. The midline nuclei of the thalamus project strongly to the mPFC, which is likely to be an important route for hypothalamic and brainstem information to reach the cortex. Afferents to midline thalamic nuclei come prominently from hypothalamic nuclei such as the paraventricular nucleus, as well as from the periaqueductal gray, parabrachial nuclei, nucleus of the solitary tract, neuromodulatory nuclei (elaborated on below), and reticular formation. The absence of widespread axon collateralization of the thalamic projection to cortex is an indication that thalamic input is topographical (Bentivoglio et al., 1993). Even axons from the pulvinar nucleus, which projects throughout many regions of cortex, do not collateralize across areas. These site‐specific projections suggest that the loops created between mPFC and thalamus are likely to have information‐specific computational roles. Neocortical‐mPFC loops 47 All three general principles of mPFC connectivity stated above can be displayed in the loops between mPFC and other regions of cortex: there is widespread, and reciprocated, convergence of input into the mPFC, with sets of regions tending to take a dorsal‐ventral or ventral‐dorsal gradient in their projections. The present summary is directly extracted from several anatomical tracing experiments (afferents: Conde, 1995; Hoover and Vertes, 2007; efferents: Sesack et al., 1989; Gabbott et al., 2005). Regions more strongly connected with dorsal mPFC tend to be areas participating in the somatic motor system. That is, the regions that are involved in computing where and how the body should be moved in relation to the environment. This includes somatosensory and motor cortex, posterior parietal cortex, and regions of visual cortex. Regions of cortex more strongly connected with ventral mPFC tend to be areas participating in the visceral motor system. In other words, the afferents are those that help identify stimuli in the environment in relation to their associations, including value and emotional significance, and the efferents also include regions involved in modulating the body and brain, discussed below. The cortical areas connected with ventral mPFC include perirhinal cortex, insular cortex, and CA1/subiculum regions of the ventral hippocampus. Only the last of these three is not strongly reciprocal, as will be discussed below. The dorsal‐ventral, “what versus where” distinction often referred to 48 with respect to the primate visual system (Ungerleider and Mishkin, 1982) can, to some approximation, be said to be preserved within these medial frontal regions (Figure 2.6). Figure 2.6 Dorsal versus ventral cortical connectivity in the mPFC. Visual processing, as with the processing within other sensory modalities, tends to follow at least two major trajectories in the brain, including computation of “where” stimuli are in the environment (dorsal pathway, blue arrow in top figure), and computation of “what” the stimuli are, or what other things are associated with the stimuli (ventral pathway, red arrow). Anterograde tracers injected into ventral sensory areas (e.g., perirhinal cortex, red circle) can be found to terminate in infralimbic and prelimbic cortex regions (red traces in the bottom coronal sections, slice locations are indicated by dotted lines in top figure). Anteriorgrade tracers injected into more dorsal sensory areas (e.g., parietal cortex, blue circle) can be found to terminate in dorsal anterior cingulate and medial precentral regions of frontal cortex (blue traces in the bottom coronal sections). Figure modified from van Eden et al. (1992). 49 The reciprocal connectivity between regions of cortex with the mPFC may not be exceptional to the frontal cortex; Felleman and Van Essen (1991) report that reciprocity throughout the visual cortex is the rule for which there are only a few (if there are any) exceptions. This may have implications for how the cortex computes: it may not be possible for cortically‐coded information to be transmitted in a way that is robust to errors without reciprocal connectivity. Feedback projections may also provide the basis for a basic computational function of cortex, such as prediction error (Hawkins and Blakeslee, 2004), or the formation of feature‐detectors. Hippocampal formation‐medial prefrontal loops The hippocampus is itself a structure defined by a loop. It is a fold of cortical tissue in the temporal lobe contiguous with the entorhinal cortex, and its primary afferents as well as efferents are with the entorhinal cortex. Together, hippocampus and entorhinal cortex comprise the hippocampal formation (although this terminology is used inconsistently, and perhaps best avoided). Superficial layers of entorhinal cortex project directly to all hippocampal subregions: layer II to the the dentate gyrus and CA3 subregion, layer III to the CA1 region and subiculum. Within the hippocampus, the dominant pathway is unidirectional: from dentate gyrus to CA3, CA3 to CA1, CA1 to subiculum. Both CA1 and subiculum project back to entorhinal cortex, with axons 50 terminating specifically in the deep layers. Deep layers of entorhinal cortex project to superficial entorhinal, as well as to superficial layers of neocortex around the brain (Witter and Amaral, 2004; Insausti and Witter, 1997). The CA1 and subiculum regions of hippocampus project robustly to the ventral mPFC (Swanson, 1981; Conde, 1991; Jay et al., 1989; Hoover and Vertes, 2007). These projections come fairly exclusively from the ventral divisions of the hippocampus (in rat; i.e., nearer the temporal pole; in monkey these would be more anterior regions). In the most densely projecting CA1 region, 20% of pyramidal neurons can be found to terminate in infralimbic cortex, 90% of which have axon collaterals that also project to entorhinal cortex (Swanson, 1981). The implication of this overlap is that that the same neurons reach the medial prefrontal cortex that, via projection to entorhinal cortex, are involved in memory formation, retrieval, and consolidation functions. Connections between hippocampus, entorhinal cortex, and mPFC are illustrated in Figure 2.7. 51 Figure 2.7 Interconnections between hippocampus, entorhinal cortex, and medial prefrontal cortex. Upper left: The posterior side of a rat brain is displayed with an overlayed, colored map of the entorhinal cortex, a line within the map separates the lateral entorhinal cortex to the left from the medial entorhinal cortex to the right. Blue areas of the color map refer to connection with more ventral regions of hippocampus, while red areas refer to connection with more dorsal regions of hippocampus. This connection pattern is depicted in the adjacent illustration of the brain (upper‐middle), which contains a similarly colored map of the hippocampus (Figures from Canto et al., 2008). Upper right: another 3‐D rendering of the rat brain, including a view of the hippocampus under the neocortex (Witter and Amaral, 2004). Green and blue squares represent horizontal and coronal slices depicted in bottom panels. Bottom: horizontal slice showing the hippocampus and entorhinal cortex (left) and a coronal slice with the prefrontal cortex (right), arrows describe connections between the regions. A direct projection exists from CA1/subiculum to the ventral medial prefrontal cortex (solid black arrow, black marks in the coronal section depict distribution of axon terminals from CA1, from Swanson, 1981). An indirect projection from hippocampus to medial prefrontal cortex also exists via the entorhinal cortex (dashed arrow), which is 52 reciprocated by the prefrontal cortex (solid red arrow). Finally, a third connection between hippocampus and prefrontal cortex exists through the mammilary bodies and anterior thalamic nucleus (dotted‐arrows). Ventral mPFC reciprocates the projection from CA1 and subiculum only very sparsely (Jones and Witter, 2007). The mPFC does, however, project strongly to the entorhinal cortex, particularly to pyramidal neurons in deep layers, matching the entorhinal projections that stem from CA1 and subicular neurons (Jones and Witter, 2007; Apergis‐Schoute et al., 2006; Insausti and Witter, 1997). This projection is to the lateral entorhinal cortex, the primary input to ventral hippocampus. The lateral entorhinal cortex is also the area of entorhinal that projects to the mPFC; thus, the entorhinal‐mPFC connection is directly reciprocal, as expected by the general patterns of neocortical connectivity. There exists a secondary route by which the entorhinal, hippocampal, and mPFC cortices are connected. This involves the projection of CA1 and subiculum to the medial mammilary bodies of the diencephalon, which further project to the anterior nuclei of the thalamus, including AM; AM is, in turn, connected with the mPFC. This brings the number of mPFC‐enotorhinal loops to three: 1) Deep lateral entorhinal cortex projects directly to superficial, and some deep, layers of mPFC, which then project back to deep lateral entorhinal; 2) Superficial lateral entorhinal cortex projects to dentate, CA3, CA1, and subiculum regions of hippocampus, CA1 and subiculum in turn project to ventral mPFC, which projects back to deep layers of entorhinal (which, in turn, project to 53 superficial layers); 3) Superficial lateral entorhinal cortex projects to hippocampus, which projects to the medial mammilary bodies, to AM, to mPFC, and back to entorhinal. The mammilary body loop was described in 1937 by James Papez, and was subsequently referred to as “Papez Circuit.” Papez believed that the circuit supported emotional functions of the brain. The visceromotor features of the ventral mPFC, combined with both the visceromotor features of the ventral striatum and amygdala to which it is connected, help validate Papez’ theory. The loops described here between the mPFC and primarily the lateral entorhinal cortex can be contrasted with the loops that involve posterior cingulate (the retrosplenial cortex), and the more prominently the medial entorhinal cortex. The posterior cingulate, dorsal‐medial entorhinal regions, and the dorsal (septal) regions of hippocampus are heavily interconnected, and are also connected by a separate pathway that passes through the mammilary bodies (Vann and Aggleton, 2004). Unlike the anatomical loops made with the ventral‐lateral entorhinal cortex, these regions are not strongly connected with the perirhinal cortex, ventral striatum, or amygdala; they are, however, connected with the postrhinal cortex and more dorsal regions of striatum. A long history of experimentation investigating the role of the hippocampus, retrosplenial cortex, and medial entorhinal cortex in spatial cognition concerns this latter loop. There are generally strong connections within, but not between, these two anatomical systems (Jones and Witter, 2007). This pattern raises the question: where and how do 54 the systems connect? As will be described below, the dual systems come into contact at several places, one of which is the basal ganglia. Basal ganglia loops The anatomy of the basal ganglia is complex. One of the more impressive reviews summarizing this complexity is written by Gerfen (2004). While it is not necessary to enter into detail on the connections of the basal ganglia, omitting them would neglect a region fundamental to mPFC physiology and function, and be a missed opportunity to relish in the aesthetic of the functional circuitry. Most of the present description is a summary of the Gerfen review, with additional details added where specified. The first step of the cortical‐basal ganglia loop is the projection of neurons in the cortex to the striatum. More ventral regions of mPFC (e.g., PrL) project to more medial and ventral regions of the striatum, which include the divisions known as the nucleus accumbens shell, and nucleus accumbens core (Voorn et al., 2004; Gerfen, 2004; Gabbott et al., 2005). Projections from dAC and mPC are to more lateral and dorsal regions of the striatum, although the most dorsal‐lateral portion of the striatum is most heavily connected with primary motor and sensory cortex. Axons that reach the striatum tend to be collaterals either from neurons projecting elsewhere in the cortex (IT‐type), or collaterals from neurons projecting to the spinal cord or brainstem (PT‐ 55 type). These collaterals differentially terminate on two different types of neurons, IT‐ type axons tending to reach striatal neurons with D1 dopamine receptors, and PT‐type tending to reach neurons with D2 dopamine receptors (Lei et al., 2004). In other words: the mPFC loop through the basal ganglia is an “efference‐copy” of cortical projections (normally reaching D1‐neurons) and descending projections (normally reaching the D2‐ neurons). Topography is preserved, in that cortical regions that are connected with one another also are those that exhibit overlap in the regions of striatum to which they project. The second step in the cortical‐basal ganglia loop are the outputs from the striatum. Depending on whether the striatal neuron is sitting in a “patch” (compartments that can be distinguished by labeling for neuropeptides and neuropeptide receptors; e.g., the mu opiate receptor; these have also been called “striasomes”), or “matrix” (non‐patch area), the feed‐forward chain of synapses either makes its way toward the dopaminergic nuclei or to the thalamus respectively. In both cases, the pathways are segregated depending on whether the projecting striatal neuron is a D1‐neuron or a D2‐neuron. The D1‐neurons are also called “direct pathway” neurons, in that they project directly to the substantia nigra pars compacta (dopaminergic; SNc) in the case of patch‐projecting neurons, or the medial globus pallidus or substantia nigra pars reticulata (SNr) nucleus in the case of matrix‐projecting neurons. The D2‐neurons, also called the “indirect pathway” neurons, project to a middle‐man region that subsequently reaches these same nuclei. In the case of D2‐ 56 neurons located in patches, axons reach the habenula, which in turn projects to the SNc; in the case of D2‐neurons located in the matrix, axons reach the lateral globus pallidus, which in turn projects to the medial globus pallidus and SNr. Together this describes the projections from dorsal striatum. Ventral striatum, which, as noted above, receives more input from ventral mPFC, exhibits the same patterns of striatal projections as dorsal striatum, but is connected with more ventral dopamine nuclei (the ventral tegmental area, VTA) and ventral regions of the globus pallidus (the ventral pallidum). In the ventral striatum, there is also a distinction between “core” and “shell” regions. This distinction adds yet more complexity to the region, in that shell and core regions have distinct projection patterns to the pallidum and to the VTA. It has been proposed that both the connection with shell to dopamine nuclei and its indirect projections into the cortex are consistent with a model in which signals spiral from core to shell (Zahm, 1999; Sesack and Grace, 2010). Note that the striatum, as well as the pallidum, are predominantly GABAergic, and the projecting neurons from one region to the next are therefore inhibitory. This adds meaning to the direct versus indirect projection system, in that the direct pathway provides inhibition of inhibition of the thalamus (i.e., excitation of the thalamus), while the indirect pathway provides three levels of inhibition, resulting in overall increased inhibition of the thalamus. The functional importance of the feed‐forward inhibitory loop, which is intimately coupled with the dopamine system that switches its signaling 57 between the direct and indirect (overall excitatory versus inhibitory) paths, will be addressed in Chapter 3 on the theories of prefrontal function. Although outputs of the basal ganglia include feed‐forward projections to the superior colliculus and brainstem nuclei, those ouputs which continue the signaling pathway that cycles back to the mPFC involves a projection from the pallidum/SNr to the thalamus (note, however, that there is also direct innervation of the mPFC by the VTA). Axons from the pallidum/SNr reach VM and MD nuclei of the thalamus as well as, more dorsally, the VA nucleus. The basal ganglia loop is illustrated in Figure 2.8. 58 59 Figure 2.8 Basal ganglia loops. A) Illustration of a coronal section of the rat frontal cortex (upper left), the thalamus (upper right), and the striatum (middle). Colors describe the mapping between projections from regions of frontal cortex, including dorsal anterior cingulate (ACd; light blue), dorsal prelimbic cortex (PLd; purple), and ventral prelimbic cortex (PLv, pink) with thalamic midline and intralaminar nuclei and with increasingly ventral‐medial regions of the striatum. Projections from the amygdala and hippocampus to these same regions of striatum are described at the bottom of the figure. Figure from Voorn et al. (2004). B) Illustration of the entire basal ganglia loop, beginning with a coronal section that includes the striatum, as in part A (main blue section). Within the striatum, both D1‐expressing direct (orange) and D2‐expressing indirect (yellow) neurons are present. Direct‐pathway neurons project to basal ganglia‐ output regions including the substantia nigra pars reticulata (SNr) and the medial globus pallidus (GPi). Indirect neurons project instead to the lateral globus pallidus (GPe), which in turn project to GPi and SNr (not shown) and to the subthalamic nucleus (STN). Output regions project to the thalamus, which project in turn back to the cortex. Due to the constraints of two dimensional space, the overlapping circuit driving dopaminergic nuclei (VTA and SNc) are not fully shown. Figure from Gerfen (1998). Subthalamic nucleus and cerebellum The subthalamic nucleus (STN) is often included in the list of basal ganglia structures, since it projects to the pallidum and the substantia nigra pars reticulata (as illustrated in Figure 2.7). Importantly, the subthalamic nucleus receives and reciprocates projections from the mPFC (Degos et al., 2008). The functional importance of this connection is still under investigation. Although the cerebellum is not a region that fits under the heading of “basal ganglia”, the circuits connecting frontal cortex, cerebellum, and thalamus share some parallels with basal ganglia loops (Middleton and Strick, 2000). Evidence from trace eyeblink studies (e.g., Takehara et al., 2000) and electrical stimulation of PrL (Watson et 60 al., 2009) suggest that the circuitry connecting mPFC and the cerebellum is functionally important for, for example, the execution of actions based on a learned expectation. Recent data suggest that the subthalamic nucleus and cerebellum are also connected (Bostan et al., 2010), suggesting a complex relationship between the anatomical loops. Amygdala‐mPFC loops The amygdala is an aggregate of multiple nuclei. Generally, the connection patterns of amygdalar nuclei are feed‐forward into the striatum‐like central nucleus, which “funnels” signals primarily to the brainstem and hypothalamus (a detailed review of the rat amygdala anatomy can be found in Pitkanen, 2000). Meanwhile, there is also a robust feedback projection to the cortex from the basal nucleus. The olfactory and entorhinal cortical regions are also reciprocally connected with the medial nucleus and cortical nuclei of the amygdala; the latter of which have their own projections into brainstem and hypothalamus. The most commonly examined pathway is from cortex to the lateral and basal nuclei, which both project to the central amygdala. The lateral amygdala additionally has feed‐forward and reciprocal connections with many other amygdala nuclei, prominent among them are the accessory basal, medial nucleus, and cortical nuclei. The accessory basal nucleus and medial nucleus join the lateral and basal in projecting to the central nucleus and are themselves reciprocally connected. The 61 medial nucleus is further reciprocally connected with the cortical nuclei, and also projects to the central nucleus. The cortical regions projecting to the amygdala are primarily those in the temporal lobe (the perirhinal cortex being particularly prominent) and mPFC. The mPFC projection primarily terminates in the basal nucleus. Infralimbic cortex (IL) projects to multiple nuclei of the amygdala‐‐the accessory basal and the medial prominent among them, but also to the anterior area, the cortical nuclei, and the posterior nucleus. The basal nucleus projects back to the mPFC throughout the ventral‐dorsal axis; the lateral nucleus projects to IL to a much lower extent. Ventral mPFC (vmPFC) neurons project primarily to excitatory neurons of the basal nuclei (cats: Smith et al., 2000); as with many cortical connections, stimulation of the vmPFC can cause an increase in the activity of the amygdala inhibitory interneurons (rats: Rosenkranz and Grace, 2001), likely through feedback inhibition. There is also evidence that the IL region projects to the intercalated inhibitory neurons, that in turn have inhibitory connections to output neurons of the central amygdala (monkey: Freedman et al., 2000; reviewed in Pare et al., 2004 and Sotres‐Bayon and Quirk, 2010). This projection may distinguish IL from PrL, along with other IL‐amygdala projections, including those to the medial amygdala and the associated bed nucleus of the stria terminals (Vertes, 2004). 62 Neuromodulatory loops The present use of “neuromodulatory neurotransmitters” refers to the synaptically‐released, short‐range molecules which terminate on receptors that are generally not ion channels; the channels are G‐protein linked, metabotropic receptors, the activation of which changes postsynaptic neuron responsivity and plasticity. Neuromodulatory neurotransmitters include the cholinergic, dopaminergic, adrenergic, and serotonergic systems. The definition provided for “neuromodulatory,” and their distinction from the glutamatergic and GABAergic systems as part of the information‐ signaling system, is often weak. Glutamate also often acts on metabotropic receptors, and acetylcholine often acts on nicotinic, cation channels (the entire skeletal muscle system being the obvious example). Numerous other molecules also act on receptors present in both mPFC neurons and in glia: short‐distance, fast‐acting transmitters such as D‐serine or the volatile free‐radical nitric‐oxide, a wide array of larger and/or longer‐ distance signaling molecules such as neuropeptides and steroids, and even molecules that are bound to cell membranes but that can evoke signaling cascades in a neuron when the two cells come into contact. However, some simplification is necessary to make sense of the system, and there are reasons to believe that the distinction between the neuromodulatory neurotransmitters listed above and the glutamate‐GABA information‐signaling system is an appropriate one. 63 Dopamine from the ventral tegmental area The ventral tegmental area (VTA) is the more ventral portion of a cluster of dopaminergic neurons that also includes, more dorsally, the substantia nigra pars compacta (SNc). Like the SNc, it is integrated into the basal ganglia circuitry described above, although it is generally linked with the ventral divisions of the striatum. In contrast to the SNc, the mPFC projects directly to the VTA, and this projection is directly reciprocated (Tzchentke, 2001). A careful examination of the neurons involved in this loop has revealed that this reciprocation is monosynaptic. The majority (~60%) of neurons projecting from the VTA to the mPFC are GABAergic, rather than dopaminergic (Carr and Sesack, 2000b). However, axons from the mPFC do not terminate on reciprocating GABAergic neurons. The GABAergic neurons that receive inputs from the mPFC instead tend to be neurons projecting to the ventral striatum. On the other hand, dopamine neurons that receive inputs from mPFC axons do project back to the mPFC (Carr and Sesack, 2000c). It is possible, then, that the same mPFC neurons innervated by dopamine, or at least those within the same local circuit, also project back to the very same dopamine neurons, creating a direct feedback loop (Figure 2.9). Generally, the VTA‐projecting neurons are concentrated more ventrally in the mPFC (Gabbott et al., 2005), but all areas of mPFC receive some level of dopaminergic innervation. 64 Figure 2.9: Connections between the mPFC and the VTA. Figure by Carr and Sesack (2000c). Projection neurons from the mPFC reach both GABAergic and dopaminergic neurons in the VTA. The dopamine neurons receiving prefrontal input can project back to prefrontal cortex (1), while the GABAergic neurons receiving prefrontal input have been found to project to the ventral striatum (nucleus accumbens; NAc). Other GABAergic and dopaminergic neurons receiving prefrontal input have been found that apparently do not project to either of the two regions (3 and 4 respectively). Signals from the mPFC also influence VTA activity in less direct ways. Axons from mPFC passing through the medial forebrain bundle reach the pedunculopontine tegmentum (PPT). This area in turn projects to the VTA, modulating its activity with both cholinergic and GABAergic signals. Dopamine exerts its influence on the mPFC via dopamine receptors, falling into “D1‐like,” (D1 and D5) receptors which stimulate cAMP production and increase excitation; and “D2‐like” (D2, D3, and D4) receptors which decreases cAMP production 65 and reduces excitation. Neurons expressing dopamine receptors of all subtypes are predominantly found in cortical layer V (Lidow et al., 1998). Both D1 and D2 receptors are located on pyramidal neurons and interneurons (Santana et al, 2009). Roughly 20% (superficial layers) to 40% (layer VI) of pyramidal neurons, and around one‐third of the GABAergic interneurons express D1 receptors. D2‐expressing neurons are found in deeper layers, with the highest proportion in layer V at about 25%. Fewer GABAergic neurons express D2 (also highest in layer V at 17%). Coexpression of the two receptor types does not appear to be higher than chance (Vincent et al., 1993; Santana et al., 2009). Interestingly, there is also no apparent overlap between neurons expressing dopamine receptors and brainstem‐projecting neurons (Gaspar et al., 1995). D3 receptor expression is more sparse in mPFC, but can be found in more abundance in regions connected to the ventral mPFC such as the ventral striatum (Bouthenet et al., 1991; Larson and Ariano, 1995; but see also Lidow et al., 1998 suggesting D3 receptor expression is equivalent between cortex and striatum in the monkey). D4 receptors have been found on inhibitory and excitatory neurons of the mPFC (Vysokanov et al., 1998) and their activation has been found to reduce the effects of NMDA‐receptor activation (Wang et al., 2003). D5 receptors have been found on the terminals of cholinergic neurons, where they may regulate cholinergic release (Berlanga et al., 2005). Acetylcholine from the basal forebrain 66 A primary source of acetylcholine to the cortex is the nuclei of the basal forebrain, to which the mPFC is strongly, reciprocally connected; primarily this connection is with the medial portion of the horizontal limb of the Diagonal Band of Broca (Sesack et al., 1989; Gaykema et al., 1990; Gaykema et al., 1991). Axons from the mPFC appear to terminate on cholinergic neurons, and topography between afferents and efferents appears to be relatively preserved (Gaykema et al., 1991). Thus, as with the other mPFC loops already encountered, the feedback system appears to be fairly specific. Projections from the basal forebrain to cortex are not exclusively cholinergic. Examination of vesicular transport proteins place glutamatergic inputs as accounting for roughly 15% of synapses from the basal forebrain, GABAergic roughly 52%, and cholinergic around 19% (Henny and Jones, 2008). However, the robust GABAergic projection tends to terminate on calbindin and somatostatin‐containing inhibitory neurons, and therefore provide disinhibition that is likely synergistic with the excitation via cholinergic and glutamatergic afferents (Freund and Gulyas, 1991). There are two classes of cholinergic receptors: the muscarinic, metabotropic receptors and nicotinic, ion‐channel receptors. Muscarinic, M1 cholinergic receptors stimulate phospholipase‐C and are most dense in superficial layers of mPFC. Muscarinic, M2 receptors, can inhibit cAMP production and are most dense in deep layers. Muscarinic, M3 receptors, also stimulates phospholipase C, and are evenly distributed throughout the layers. Nicotinic cholinergic receptors are higher in layers I‐II and V, while II shows extremely low values. 67 Many nicotinic receptors are thought to be found on the terminals of thalamic glutamatergic projections, and thus modulate the thalamic input into the mPFC (by modulating glutamate release at presynaptic terminals; Gioanni et al., 1999; Lambe et al., 2003); although, this effect may not be selective to synpases carrying input signals: nicotinic receptor stimulation also appears to increase glutamatergic neurotransmission of intrinsic fibers. Norepinephrine from the locus coeruleus Norepinephrine is the primary neurotransmitter used by the sympathetic nervous system, and its reach extends throughout the brain and body, in large part determining the animal’s state of arousal. Norepinephrine in delivered throughout the forebrain by the locus coeruleus (LC; Foote et al., 1983). Its projection patterns are not topographical, but highly diffuse, and this may distinguish its loops with frontal cortex in comparison to other neuromodulator systems. Primary afferents to the LC come from brainstem nuclei, with some projections also coming from the hypothalamus (Luppi et al., 1995); although influences may also come from the bloodstream (Foote et al., 1983). There exist direct projections from the ventral mPFC (Luppi et al., 1995) and from mPC cortex (Sara and Herve‐Minvielle, 1995; Arnsten and Goldman‐Rakic, 1984, have also reported projections from the monkey prefrontal cortex). However, these projections are sparse, and the influence of the 68 mPFC on the LC likely is largely indirect, via its brainstem and hypothalamic afferents (Gabbott et al., 2005). Norepinephrine from the LC acts back on the prefrontal cortex through alpha‐1, alpha‐2, and beta receptors. Alpha‐2 receptors are high‐affinity receptors that inhibit cAMP production, and their inhibition has been shown to improve working memory on cognitive tasks. Alpha‐1 receptors and Beta receptors are lower affinity, the former stimulates phospholipase C, the latter stimulates cAMP production. The varying affinities of receptor types are thought to contribute to the “U‐shaped curve” observed in the relationship between arousal and performance (reviewed by Arnsten, 1998). Alpha1 receptors are highest in layers I‐II and V, while Alpha‐2 receptors, relatively high in the mPFC, are highest in superficial layers. Although the mPFC‐LC loop appears less direct and more diffuse than loops made by the mPFC with other brain systems, this does not diminish its importance. The mPFC has direct access to controlling LC activity, and therefore may be only a synapse away from influencing the state of neural communication throughout the entire telencephalon. Serotonin from the dorsal Raphe nucleus 69 The projection from the mPFC to the dorsal Raphe nucleus (dRN) is exceptionally strong, and is less dominated by a ventral‐dorsal gradient than many of the connections with other brainstem nuclei (Gabbott et al., 2005). The specifics of the mPFC‐dRN projection differs, however, from the mPFC‐VTA connection. As a reminder, in the mPFC‐VTA connection, descending mPFC pyramidal neurons were found to synapse directly on dopamine‐containing neurons that returned the projection, but not the returning GABA projections. In contrast, mPFC‐dRN axons are more likely to terminate on GABAergic neurons than serotonergic, the GABAergic neurons are then thought to project locally to inhibit the serotonergic neurons (Jankowski and Sesack, 2004). The projection from mPFC to the dRN is also primarily from PrL and more dorsal regions, with fewer dRN afferents originating from the IL (Gabbott et al., 2005) The return projection from the Raphe nucleus is extensive throughout cortex, with a particularly high number of serotonin‐receptor expressing neurons in mPFC. There are many serotonergic receptor classes, and many subtypes within those classes. Moreover, serotonin likely exerts its influence not only post‐synaptically, but also presynaptically, both via self‐regulation at autoreceptors, and potentially by influencing the release of other neurotransmitters such as acetylcholine (Buhot, 1997). Two highly expressed, postsynaptic serotonin receptors are the 5‐HT1A (which inhibits cAMP production) and the 5‐HT2A (which stimulates phospholipase C). Each of these receptors can be found on the majority (50‐70%) of pyramidal neurons, with 80% colocalization between the two (Santana et al., 2004). Each can also can be found on 70 20‐30% of inhibitory neurons. Receptors that are often found presynaptically, including on cholinergic terminals, additionally include 5‐HT4 and 5‐HT3 receptors, which stimulate production of cAMP (Buhot, 1997). The robust reciprocal connection between the mPFC and Raphe nucleus points to serotonin as a powerful mediator of mPFC activity. 7 Owing to the internal circuitry of the dRN, and the complex distribution of serotonin receptors in the mPFC, the dynamics of this loop are highly complex, and not well understood. Non‐neurotransmitter modulators Although not discussed in detail presently, the mPFC contains receptors for a number of non‐neurotransmitter biological factors, including peptides. For example, the mPFC is a major site of oxytocin receptor binding, which is present in both monogamous and non‐monogamous rodent species (Young and Wang, 2004). The mPFC is also major site of opioid receptor expression, particularly delta and mu types (Mansour et al., 1994). Loops with the hypothalamus and brainstem 7 This is consistent with the fact that drugs that act on the serotonergic system comprise most of the dispensed mental health‐related pharmaceuticals in North America (see Hhttp://www.rxlist.comH source of data is IMS Health, a private institution specializing in gathering health‐market related information). 71 Anatomical tracer studies find a large efferent projection from the entire dorsal‐ ventral extent of the mPFC to the lateral hypothalamus, originating predominantly in layer V (Gabbott et al., 2005). The lateral hypothalamus also, to some degree, appears to reciprocate this projection back to the mPFC (e.g., Hoover and Vertes, 2007). An important consideration when considering these anatomical data is that the medial forebrain bundle passes through the lateral hypothalamus, carrying axons from the mPFC to midbrain regions such as the VTA and pedunculopontine tegmentum. Infralimbic cortex (IL) also exhibits projections to other hypothalamic nuclei, including the anterior nucleus. IL signals also reach the hypothalamus indirectly, by projecting first the lateral septum. The lateral septum exhibits highly‐organized reciprocal connectivity with the hippocampus (including, uniquely, the CA3 subfield), and heavily projects to the hypothalamus (Risold and Swanson, 1997). The projections from mPFC to brainstem regions such as the parabrachial nucleus and pariaqueductal grey (PAG) tend to be stronger from more ventral regions of mPFC (Gabbott et al., 2005). The mPFC also projects to the superior colliculus, which may follow a more dorsal‐ventral gradient (Wyss and Sripandkulchai, 1984; Figure 2.10). The superior colliculus also exhibits a strong projection back to the MD nucleus of the thalamus (e.g., Kuroda et al., 1998), completing a loop between the mPFC and brainstem. In Chapters 10 and 11 of this thesis, physiological data will be examined that speaks to neural signaling about approach to a specific location or a withdrawl or 72 slowing down under conditions of doubt, which may engage these regions of the brainstem. Figure 2.10 Dorsal‐ventral gradient of mPFC projections to the midbrain. As observed in other connection patterns to the mPFC, more dorsal regions such as the medial precentral cortex (PC) and the dorsal anterior cingulate (dAC; blue circle in top, unfolded sagital section of the cingulate, represents site of radioactively‐labeled amino acid 73 injection) are more strongly connected with more dorsal regions of the midbrain, including the superior colliculus (SC, s = superficial later, i = intermediate layer, p = deep layer; blue circles in bottom, coronal section of midbrain, represent termination sites of dorsal anterior cingulate cortex axons). More ventral regions of mPFC, such as infralimbic cortex (IL; DP = dorsal pallidal cortex ventral to IL) are connected with more ventral‐medial regions of the midbrain, such as the periaqueductal gray (PAG, m = medial, l = lateral; red circles in bottom section represent termination sites of infralimbic and dorsal pallidal cortices; purple circles represent termination sites of prelimbic cortex, PL). Figure modified from Wyss and Sripandkulchai (1984). Endopiriform nucleus and claustrum As pointed out in the sections on amygdala and hypothalamus, although ventral prelimbic cortex and infralimbic cortex share a number of connections, and are themselves connected, there are also a number of differences between the regions (Vertes, 2004). Among the more interesting patterns that distinguish the two are the ways in which each is respectively connected to the claustrum and endopiriform nucleus. In the past the claustrum and endopiriform nucleus were considered part of the same region, though they are different developmentally, cytoarchitechtonically, and in their connection patterns with other regions of the brain (Behan and Haberly, 1999). The claustrum is reciprocally connected to all areas of cortex, almost as though it serves as an additional cortical layer, and Koch and Crick (2005) have proposed that its connectivity patterns make it a candidate substrate for consciousness. The endopiriform cortex, on the other hand, projects throughout the “limbic system,” to the 74 piriform cortex, entorhinal cortex, perirhinal cortex, insular cortex, and orbitofrontal cortex, as well as the hippocampus proper (Behan and Haberly, 1999). Chapter summary The rodent medial prefrontal cortex (mPFC) is used to describe the dorsal‐to‐ ventral frontal regions that include the medial precentral cortex, the dorsal anterior cingulate cortex, the prelimbic cortex, and the infralimbic cortex. These regions appear to be homologous with subregions of the primate medial frontal cortex (e.g., overlapping with Brodmann’s regions 24, 32, and 25 in the human respectively, but better articulated by connectivity‐based parcellation methods as in Beckmann et al., 2009). There is a large degree of controversy regarding whether the rodent mPFC also shares homology with the primate dorsolateral prefrontal cortex; this chapter and Chapter 3 argue that evidence for this claim is weak. Within the rodent mPFC, a number of neuron types can be found that generally fall into the classes of excitatory pyramidal neurons and inhibitory interneurons. Multiple different classes of inhibitory neurons can be identified that likely have different functions for local circuit activity, discussed in greater detail in Chapter 4. Connectivity of the mPFC exhibits a strong dorsal‐ventral gradient. Dorsal prelimbic and dorsal anterior cingulate cortices are connected, and infralimbic cortex and ventral prelimbic cortex appear to be connected. Dorsal regions are more strongly connected with parietal and motor cortices, while ventral mPFC is 75 more strongly connected with more ventral regions of cortex such as the anterior insula and perirhinal cortex. The dorsal‐ventral gradient is reflected in a dorsolateral‐ ventromedial gradient of projection patterns to the striatum, which provides the first step of the basal‐ganglia loop that ends in the thalamic nuclei to which the mPFC is connected. Of all thalamic nuclei, the mPFC is most strongly connected with the medial dorsal nucleus (MD). The mPFC, particularly the most ventral division, the infralimbic cortex, is strongly connected with the ventral hippocampus. This region receives direct input from the hippocampus, is reciprocally connected with the divisions of entorhinal cortex that are connected with ventral hippocampus, and receives indirect hippocampal input through the mammilary bodies. Ventral mPFC tends to be more strongly connected with hypothalamic and brainstem regions involved in controlling visceral and emotional states (e.g., the periaqueductal gray), and is also indirectly connected to these regions through the amygdala, while more dorsal mPFC regions tend to be connected with regions of the brainstem involved in movements, such as the superior colliculous. Finally, the mPFC contains many loops with neuromodulatory systems including dopaminergic, serotonergic, adrenergic, and cholinergic systems. These loops are sometimes more specific to individual neurons or microcircuits than might be expected by global network modulators, and likely play a large role in determining the state of neural processing in the mPFC, also discussed at more length in Chapter 4. 76 CHAPTER 3: BEHAVIORAL, COGNITIVE, AND EMOTIONAL FUNCTIONS OF THE MEDIAL PREFRONTAL CORTEX Chapter aims and challenges What good is a medial prefrontal cortex? In this chapter, theories and data on the psychological functions of the cingulate regions, or medial prefrontal cortex (mPFC) will be reviewed. This chapter does not discuss involvement of the mPFC in behavioral and cognitive changes that accompany pathological states, such as schizophrenia, Alzheimer’s, and depression. For the most part, the specific circuits that transmit specific information to the mPFC, such as a prominent pain pathway, will be considered only under the broader headings of “sensory information” and “value information.” It may be the case that prefrontal function can not be fully understood without these details, in which case the present chapter will fail in its attempt to outline what the mPFC does. This review will begin with an overview of how the prefrontal cortex is thought to work. It will then examine converging evidence that the dorsal regions of mPFC participate in the decision‐making process by explicit representation of action value, or association with action outcomes. The review of function then moves ventrally into the 77 prelimbic region, identifying it as a composite of both dorsal, action‐selection systems and ventral, emotion selection systems. More ventrally still, the functions of the infralimbic cortex and connected regions of ventral prelimbic will be discussed. A theme throughout the discussion of the cingulate subregions is “expectation,” how expectancies are coded by the cingulate region to allow it to perform the functions that it performs. In view of this, the final sections of the review discuss first the functional interactions between the hippocampus and prefrontal cortex in building or retrieving expectation, and second the importance of prefrontal interaction with the dopamine system for training expectation (this latter topic is continued in Chapter 4 on the role of the mPFC‐dopamine loop not only for training networks, but also for dymamically setting the network state). The efforts to develop theories for prefrontal function have encountered, and still follow, a difficult road. With respect to nervous system connections, the prefrontal cortex is as far removed from sensory and motor systems as any region is. It has been associated with “higher order functions,” but the process of identifying those functions has required creativity in explaining why subregions of prefrontal cortex seem to be involved in every task an animal performs, and at the same time, no particular subregion appears to be required for an animal to perform any task. Mechanistic theories have been developed in parallel with descriptive theories. This is useful, in that gaining insight into the function of a system can be aided by putting together the system’s parts, observing how they interact and what they do. It can also add to the confusion, 78 as mechanistic theories can begin to shift the study of prefrontal function from a reverse‐engineering enterprise to one of engineering. The difficulties faced by prefrontal theories have influenced the degree to which the proposed theories are falsifiable. For examples, the terms “inhibition”, “conflict detection” and “cognitive control” have all been associated with mPFC function, but have not always been presented in a way that is concrete and complete enough to empirically contradict. A more specific example is the controversial role played by the mPFC in expression of memory associations that initially depend on the hippocampus. Debates revolving around the physiological mechanisms for the observed behavioral phenomena have developed without either side concretely articulating what those physiological mechanisms are. That is: what does it mean, in the end, to retrieve a memory? For that matter, what does it mean for a brain region to contribute to decision‐making, and can any region of the brain be said not to contribute? Despite the challenges, enormous progress has been made in articulating the functions of the mPFC; what follows is one perspective on that progress. General theories of prefrontal function Earlier theories of prefrontal function rely heavily on the consequences of accidental damage to the human frontal lobe. Throughout the 20th century, a number 79 of case reports have been published that link prefrontal function with behavior and cognition (reviewed by Fuster, 2008). Not unexpectedly, the consequences of damage are highly dependent on the particular locus of insult. This is well reflected by three, widely‐known historical examples 1) the increased emotional instability of Phineas Gage following ventro‐medial prefrontal damage; 2) the apparent “calming” and personalty “blunting” observed following the widely‐practiced medical procedure of frontal lobotomy; 3) Brenda Milner’s use of specific cognitive batteries to identify selective deficits in frontal patients, notably, deficits in switching between task rules (attentional sets) in the Wisconsin Card Sorting test following dorsolateral frontal damage (Milner, 1963; 1982). Joaquin Fuster developed an integrated theory of prefrontal function by placing it into the context of Sherrington’s hierarchy of reflex‐arcs, as described in the introduction of this thesis (Fuster, 1980). To Fuster, the prefrontal cortex enables behavior to be temporally organized, by serving working memory functions and by representing action gestalts, or “schemas.” Working memory refers to the ability to temporarily maintain and support information (Baddeley, 2003), which has been linked to persistent neural activity during delay periods (Fuster and Alexander, 1971; Fuster et al., 1995). Of schemas, he writes: “The schema stands for the plan or program of action. It does not represent all its elements and steps, however. It is an abridged, abstracted, representation of that plan or program, which may contain some of its components and also contains, in some manner, its goal”(Fuster, 2008; p345). The idea that schemas are 80 represented by prefrontal cortex is consistent with data showing that neural activity in prefrontal correlates with categories of action sequences, rather than the individual actions or their parts (Shima et al., 2007). Another general theory of the prefrontal cortex has been developed by Todd Braver, Joel Cohen, and Randy O’Reilly (Cohen et al., 1996; Braver and Cohen, 2000). In a widely read review article, Miller and Cohen (2001) present its principles, that the prefrontal cortex could support an individual’s ability to use dynamically‐learned and contextually‐dependent goals to quickly overcome learned or “pre‐potent” behaviors. This theory highlights in particular two sets of anatomical loops described in Chapter 2: the loop between prefrontal cortex with sensory and motor cortex, through which the prefrontal cortex exerts its influence, and the loop the prefrontal cortex has with the dopamine system providing a reinforcement signal. Not unlike Fuster’s temporal‐ control‐of‐behavior theory, the model posits that the prefrontal cortex contains representations of action/attentional sets (rules and schemas). Normally, the neurons representing these action sets are either quiet or fire according to the behaviors an individual is performing. However, these representations are highly modulated by the dopamine reinforcement signal. When an individual encounters a stimulus, or performs a behavior, which is linked to the discharge of dopamine (i.e., when the subject encounters an unexpected primary or secondary reinforcer), the set of prefrontal neurons linked to that stimulus or action begin to fire. Working memory is established through the intrinsic ability of these neurons to continue firing, and therefore to 81 continue influencing networks in sensory and motor cortex. In the model, an absence of reinforcement signal, such as during the omission of an expected reinforcer, can reduce or eliminate the persistent activity of the prefrontal neurons and promote exploration of alternative action sets. Predictions made by the Braver and Cohen model are consistent with those made by Fuster, with the addition of the incorporation of interactions between the prefrontal cortex and dopamine (illustrated in Figure 3.1). Figure 3.1: Schemas and cognitive control. Left: Fuster has claimed that the prefrontal cortex (the top of the motor hierarchy, outlined here in a purple square) represents a top‐layer of motor‐processing, representing action sets (schemas) that can affect both sensory processing and the particular actions that are selected (figure from Fuster, 2001). The sensory‐motor hierarchy is described by the ascending levels of sensory processing in posterior regions of cortex (left) and ascending levels of motor processing in anterior regions of cortex (right). Actions then act on the environment, causing changes in sensation and completing the cycle. Right: Cohen and collaborators have described the prefrontal cortex (upper‐right “control” oval, outlined here in a purple square) as providing a top‐down signal encoding learned goals or rules that biases sensory processing of environmental stimuli (shown here in blue; for simplicity, prefrontal connection with action‐signals are left out of the figure; figure from Miller and Cohen, 2001). In this figure, incoming sensory input of the word “GREEN,” colored 82 in red, is presented to the system (bottom oval). Because of strong connection strengths of word‐stimuli, in comparison to color‐stimuli, onto higher‐order sensory processing systems (middle oval, connection strengths shown with thicker lines), the stimuli would normally evoke a verbal response “GREEN”. In this example, the system has been told to pay attention to colors, so the “color” rule‐encoding neurons of the prefrontal cortex (upper‐right oval) are active, causing increased activity of the “red” color‐coding neurons in higher‐order sensory areas (middle oval again). Therefore, the action that is selected is the verbal response “red” instead of “green”. For simplicity, the connections between prefrontal cortex and lower‐levels of the motor system are not shown, but these connections could equally effectively influence the selected verbal response. One other theory of prefrontal cortex is worth mentioning that is also compatible with both of the frameworks described above. This is the hypothesis by David Gaffan (2002) that the prefrontal cortex can be considered the top of a hierarchical scheme, and, at the top, this is where specialization of function breaks down. While damage to localized regions of prefrontal cortex may temporarily elicit specific behavioral deficits, it is often the case that behavior recovers, as though compensatory mechanisms come into play. Gaffan claims that a general absence of double‐dissociations between damage to separate regions of prefrontal cortex supports the conclusion that the prefrontal cortex is only specialized for the variables that an animal has learned are behaviorally relevant: To support unpredictable processing demands it is necessary to abandon local grouping after a certain stage and allow neurons to interact at random. It is unsurprising in this context that it is the animals of highly intelligent and adaptable species—humans, apes and Old World monkeys—that alone possess a large area of prefrontal cortex. Certainly this would also explain why conditional discrimination tasks, in which animals 83 learn quite arbitrary rules, are so sensitive to the effects of prefrontal cortex lesions in macaques (Gaffan, 2002). How do these theories of schemas (Fuster), cognitive control (Cohen and colleagues), and distributed processing (Gaffan) account for the diversity of effects of prefrontal damage? Of the three historic examples noted above, both Fuster’s ideas of schemas and cognitive control theories are built to account for deficits observed on the Wisconsin Card Sorting Task (particularly the set of models by Cohen, Braver, and O’Reilly). Explanations of Phineas Gage’s instability, or the “deadening” of personality following frontal lobotomy, are not directly made by the theories. However, they also are not incompatible, and may emerge by extending top‐down control signals to also include emotional schemas and action sets. As a final note on general theories of prefrontal cortex, the question “What function does the prefrontal cortex serve as a whole?” is inextricably linked to the question “What computational function does a given region of prefrontal cortex serve for the inputs and ouputs to which it is connected?” There are few scientists who would claim that a synapse between a pyramidal neuron and fast‐spiking parvalbumin neuron found in one region of cortex has a different computational meaning than an equivalent synapse found in another region. But at what point does the ability to generalize break down? The idea that a cortical column represents a cortical module has been examined 84 by Vernon Mountcastle (reviewed by Mountcastle, 1997). 8 Also, the “inter‐ changeability” of cortical modules has empirical support, in that axons from the thalamus targeting one region of neocortex during development can be re‐routed to innervate another, with some preservation of function (although this re‐routing also causes a number of changes in the target region; reviewed by Sur and Learney, 2001). These issues will be returned to in the next chapter on physiological computation in the cortex. What does the dorsal mPFC do for behavior and cognition? Examining how action‐ outcome expectations are represented and compared. There is clear distinction between the functions of more ventral and more dorsal regions of the medial prefrontal cortex (vmPFC and dmPFC respectively). This has been reviewed in some detail in the human and monkey (Vogt et al., 1992; Bush et al., 2000; Beckmann et al., 2009), and has been specifically addressed by a number of lesion studies in the rat (a partial set include: Seamans et al., 1995; Morgan and LeDoux, 1995; Kesner et al., 1996; Fritts et al., 1998; Ragozzino and Kesner, 1998; Walton et al., 2003; Killcross and Coutureau, 2003; Tzschentke and Schmidt, 1999; Delatour and Gisquet‐ 8 Some of the theories that link local, anatomical connectivity in the cortex to the cortical region as a processing module depend on the observation that the neurons firing in a given cortical column have correlated firing properties. It should be noted that methods which use advanced techniques to distinguish neuron action potential have found evidence for active decorrelation within a local region of cortex (Renart et al., 2010 and Ecker et al., 2010) 85 Verrier, 2001; Passetti et al., 2002; Dalley et al., 2004; Marquis et al., 2007; St Onge and Floresco, 2009). There is also evidence of specialization within dmPFC between dAC, dorsal PrL, and mPC regions; and evidence of specialization within ventral mPFC regions, between IL and ventral PrL regions. In most rodent PrL lesion studies, both ventral and dorsal divisions of PrL are damaged, making interpretation of these specializations difficult. A number of studies have identified the dmPFC as an area that may be involved in calculating the value of performing a given action. When the dmPFC is lesioned, experimental subjects become impaired at choosing the action associated with the highest value (monkeys: Hadland et al., 2002; Buckley et al., 2009; rat: Walton et al., 2002; Walton et al., 2003; Schweimer and Hauber, 2005; Rudebeck et al., 2006; Walton et al., 2009). The dynamically‐changing value of actions can be dissociated from the dynamically changing value of environmental stimuli, as damage to the anterior cingulate does not impair selection of stimuli associated with the highest reward (e.g., DeCoteau et al., 1997); learning and representing the value of stimuli is a function that is associated with the orbitofrontal cortex and the amydala (e.g., Baxter et al., 2000). In one notable experiment, AC lesions in monkeys had no effect on the ability of the monkey to switch to a different action when the old action stopped yielding rewards; lesions did, however, cause the monkey to be less likely to persist in using the newly‐rewarded action (Kennerley et al., 2006). These data could be interpreted in 86 several ways. One likely possibility is that, although the lesioned monkey’s actions could still be guided by omission of reinforcement, the omission of reinforcement for one action and/or subsequent reinforcement of the alternative action did not appropriately update the relative values of the two actions. Alternatively, the individual values were updated, but could not be maintained and subsequently compared. Many of the cited rat lesion studies are show that dmPFC damage reduces the likelihood that a rat will work more for higher reward, such as choose to climb a higher barrier (e.g., Walton et al., 2002, 2004, 2009; Rudebeck et al., 2006). Again, there are multiple possible explanations for why decisions are altered following dmPFC damage, but some of these are consistent with the interpretation in the primate study described above: it is possible that the relative differences in outcome value and action costs between the two actions could not be weighed against one another. The effects of lesion on rodent spatial decisions can not be fully explained by a complete loss of action‐value associations: when the climbing barrier was removed, the lesioned rats resembled controls in preferring the higher‐reward location. The apparently normal behavior of the lesioned rats when the action costs are equalized may depend on other value‐representation systems in the brain that take part in decision‐making, and which could compensate for the absence of action‐value processing in the dmPFC. Also consistent with the idea that the dmPFC represents action value is the observation that damage to the rat dmPFC causes deficits in an action‐based, delayed‐ 87 non‐matching to sample task (Kesner et al., 1996; Ragozzino and Kesner, 2001) and delay match to sample tasks (Dias and Aggleton, 2000; however, see Walton et al., 2003, who find no deficit with anterior cingulate lesions on a match‐to‐sample task). Such deficits could be related to action‐memory, or alternatively the inability of the rats to dynamically update the values of their prospective actions. Electrophysiological data has also been strongly supportive of the relationship between action and value in the dorsal anterior cingulate. Recordings have revealed a predominance of activity patterns that relate both to actions and to rewards, as well as, very frequently, their combination (monkey: Shidara and Richmond, 2002; Matsumoto et al., 2003; Amiez et al., 2006; Quilodran et al., 2008; Sallet et al., 2007; Kennerley et al., 2009; Kennerley and Wallace, 2009; rat: Cowen and McNaughton, 2007; Euston and McNaughton, 2006; Hillman and Bilkey, 2010). Importantly, the electrophysiological experiments reveal that neurons are correlated with more than just the combination of reward and movement. Very often, neurons fire in relation to the expectation of reward, and the error in its prediction (reviewed by Rushworth et al., 2008). The reward signal is also dependent on the perceived value of outcomes from alternative choices (Hayden et al., 2009). The link between actions and values in the anterior cingulate cortex, or dmPFC, fits into a larger set of decision‐making theories that distinguishes between actions that are selected based on stimulus‐response systems (habits) and those that are selected 88 based on association of the actions or stimuli with specific outcomes. The habit and action‐outcome systems learn at different rates (habit being associated with overtraining, action‐outcome with new training) and are dissociable though specific experimental manipulations, such as omission of a reinforcer, which has larger effects on behavior while it’s under action‐outcome control (reviewed in Dayan and Balleine, 2002). Killcross and Coutureau (2003) found that lesions to the infralimbic cortex (IL) region affected habit behavior, while lesions to prelimbic cortex (PrL) selectively impaired action‐outcome associations. Although PrL is often associated with the vmPFC, its connectivity patterns suggest that it can be separated into dorsal and ventral subdivisions, with the dorsal better resembling dAC (Neafsey et al., 1993), which the Killcross and Courtureau lesions appeared to be centered on. Contrasting data comes from Corbitt and Balleine (2003), who show that prelimbic lesions do not affect action‐ outcome associations (based on lever‐pressing rates following reinforce degradation). However, the PrL rats had an overall decrease of lever‐pressing in the task, and, unlike controls, did not discriminate between the degraded and non‐degraded levers. Although lesions in this study are selective and cover most of ventral and dorsal regions, they do appear to spare more posterior portions of the PrL, and may infringe on the more ventral IL. Theories of mPFC function have been built not only on behavioral and electrophysiological data, but also on models of learning and action selection. This will be elaborated on below with respect to the loops the mPFC shares with dopamine and 89 the basal ganglia, but it also applies to the observed differences between action‐ outcome and stimulus‐response learning. Daw, Niv, and Dayan (2005), for example, have associated outcome‐dependent decisions made by the mPFC with a “tree search” algorithm, by which the rat can explore the consequences of various available alternatives. This concept is not different from the idea of “vicarious trial and error,” a term coined by Muenzinger to describe a phenomenon also discussed by Tolman (Muenzinger, 1938; Tolman 1948), in which the rat pauses at a choice point to assess the relative benefits of its options. 9 The addition made by Daw et al. is that when the action‐outcome signals are in conflict with stimulus‐response signals, the brain can select one or the other according to the uncertainty of each. The exact manner by which neural systems might do this remains unclear. Several studies have implicated the dmPFC in the ability to chain a sequence of actions together (e.g., Passetti et al., 2002; Ostlund et al., 2009). This is a function commonly associated with the supplementary motor area (SMA) and pre‐ supplementary motor area (pSMA) in humans and primates (Goldberg, 1985; Tanji, 2001; Shima and Tanji 1994, 2000; Jenkins et al., 1994; Nakamura et al., 1998). The lesions that are performed in the rat studies include more dorsal, medial premotor cortex above the cingulate, and extend far enough posterior to include the rat homologue of SMA and pSMA. 9 VTE has been “reconsolidated” in the neuroscience field by recent data from the Redish lab showing that hippocampal and striatal activity transiently reflect goal points while rats slow down at a choice point (Johnson and Redish, 2007; van der Meer and Redish, 2009). 90 Relationship between cognitive control and action selection In 2001, Botvinick and colleagues published a review that linked activity in the dorsal anterior cingulate with conflict monitoring (Botvinick et al., 2001; Botvinick and Carter, 2004). The novel insight of this theory was that, rather than fitting the anterior cingulate into the hierarchy of action‐perception cycles set‐up by Sherrington, elaborated‐on by Fuster, and articulated computationally by Coen and colleagues (illustrated in Figure 3.1), they associated the anterior cingulate with a unique function: providing a bottom‐up signal to other prefrontal regions under conditions of decision‐ conflict. The theory addresses a large body of literature that suggests the anterior cingulate becomes more active when there is increased ambiguity in the factors influencing a person’s behavior. The specific conditions of ambiguity include 1) when an experimental subject must respond in a way that “overrides” a previously‐learned or automatic response, 2) when the subject must select a response from a set of responses that are apparently equal in value, and 3) following, or immediately preceding, a wrong response. (Although the last of these is not intuitively linked to monitoring response conflict, the authors cite several studies they believe associate the two, including the observation that activity associated with errors is frequently followed by reversals; i.e., that the activity increase still reflects the simultaneous representation of both action sets). In this theory the anterior cingulate (dmPFC) receives information from sensory 91 and motor systems, and the activity of cingulate efferent neurons become activated when these responses are in conflict with one‐another. These signals then reach other frontal cortical regions (e.g., dorsolateral prefrontal cortex) or modulatory nuclei (e.g., dopaminergic or adrenergic) to increase the level of cognitive control, as described in the preceding section. The expression that the anterior cingulate cortex is “more active” during conditions of response conflict is short‐hand for measurements of 1) increased bloodflow to the region, often using functional magnetic resonance imaging which provides a blood‐oxygenation level dependent (BOLD) response, suggesting increased energy‐consumption and therefore neural activity; or 2) evoked, scalp‐recorded electrical potentials (event‐related potentials, or ERPs), which provide an indication of changes in neural synchrony in the underlying neocortex. BOLD responses can be localized with relatively higher degree of spatial precision, but are limited in their temporal resolution (which can be partially overcome using event‐related paradigms). ERPs have a high degree of temporal resolution (although still require event‐related paradigms, as they rely on signal averages across repeated episodes), and localization of ERP signals can only be roughly approximated, based on the distribution of the signal across different scalp electrodes. When individuals are in the process of committing errors, a distinctive ERP and event‐related BOLD signal can be observed, localized to the anterior cingulate cortex, called the error‐related negativity (ERN; a brief history of research on the ERN can be found in Falkenstein et al., 2000). Two ERPs exhibit similar 92 structures to the ERN: the feedback‐related negativity (FRN), which resembles the ERN but takes place following negative feedback about an action (Simons, 2010), and what has come to be called the “conflict” N2, a negative‐going ERP taking place 200ms after presentation of cues that may elicit response conflict (van Veen and Carter, 2002). The circuit‐level dynamics that may be responsible for the negative‐deflecting ERP signals will be revisited in Chapter 4. The ERN, FRN, and conflict N2 ERP signals have been argued to represent the competition, or conflict, between more than one action (Yeung et al., 2004). This may initially seem counter‐intuitive, in that the error activity follows the execution of the decision, and therefore, in principle, should not be related to conflict between the actions themselves. However, as Yeung and colleagues argue, the ERN can be explained as a mismatch between the action that is being executed and an alternative action that neural activity patterns are instantiating as the potentially correct action. To take these ideas further, it might be hypothesized that the FRN represents a mismatch between the remembered executed action and an activated “vicarious” correct action (neural activity related to the correct action would be useful for an association to be formed between the correct action and the situational factors of the trial and task). By the conflict‐monitoring hypothesis, this conflict signal influences the “higher‐ level” centers of prefrontal cortex to drive top‐down signals onto perceptual and/or motor areas, as presented in Figure 2.1. This contrasts the view described in the 93 previous section, that the cingulate activity does not play a monitoring role but an active role in representing and executing the alternative actions and their respective values. In this view, the cingulate‐generated ERN is less of a signal to higher‐level regions about the need for cognitive control, and instead represents active processes that may be taking place in immediately correcting the decision. Classic work by Rabbitt (reviewed by Yeung et al., 2004) showed that fast, automatic error correction was more likely to occur if subjects were able to continue processing decision‐cues. Support for the hypothesis that the cingulate is responsible for fast, automatic error correction comes from a study showing that patients with damage to the anterior cingulate are slower to correct their errors (Modirrousta and Fellows, 2008b). Notably, this study, and an earlier study from the same group, also found that subjects were just as likely to adjust behavior following errors, i.e., to exhibit “post‐error slowing” which has been thought of as the implementation of increased cognitive control (Fellows and Farah, 2005; Modirrousta and Fellows, 2008b). The conflict‐monitoring hypothesis is not mutually exclusive with the apparently direct role the medial prefrontal cortex plays in selecting behaviors or emotions, based on their values or appropriateness in a specific context. “Cognitive‐control regions” is often used to refer to dorsolateral prefrontal cortex, a region that is not clearly present in rodents (Chapter 2). It could be that the evolution of lateral prefrontal cortex regions represent additional layers of control over that of the cingulate, and that the 94 engagement of these regions follows, and may even depend on, the engagement of the anterior cingulate cortices. Two notable examples that explicitly tested the role of the rat dmPFC in conflict resolution include de Wit et al., (2006), who find that dmPFC lesions impair the ability of rats to make choices under conditions of response conflict, and Haydon and Killcross (2004), who find that mPFC damage impairs the ability of rats to use a contextual cue to determine which of two conflicting cue sets to attend to (see also Marquis et al., 2007, who show this ability is more strongly affected by lesions to the PrL than IL). These results can be integrated within the framework of dmPFC representing the context‐ dependent and dynamically‐changing values and associated outcomes of actions. The connection between conflict‐resolution and conflict‐monitoring in rodents remains to be examined. The ugly duckling prelimbic cortex: part dmPFC, part vmPFC, living in a dorsal‐lateral PFC world For historical reasons, PrL is often thought of as a less‐differentiated homologue of primate dorsolateral prefrontal cortex (see the first section of Chapter 2). Because of this, many of the early studies examined the effect of mPFC lesions on tasks that require the maintenance of memory during a delay (“working memory”). Confirmation biases in interpreting the results of these studies have perpetuated the belief that the region is a 95 homologue of primate lateral cortex; meanwhile, deficits on delay tasks in primates have also been observed with frontal lesions outside of dorsolateral prefrontal cortex (reviewed in Preuss, 1995). A careful examination of the rodent delay tasks reveals that deficits can be accounted for by explanations other than working memory impairments. For example, in some tasks rats are offered the option to re‐visit a specific location or object where the reward had already been obtained. Rats with vmPFC damage are more likely to take this option (Ragozzino et al., 1998; Ragozzino et al., 2002). While these errors can be interpreted as working memory impairment, a likely alternative is that the vmPFC rats exhibit increased impulsivity toward whatever option is offered, which has also been associated with vmPFC damage (Chudasama et al., 2003). This explanation does not account for the results of all working‐memory task experiments; however, this does not make these other results immune to alternative explanations. Gisquet‐Verrier and Delatour (2006) found that the effect of PrL damage on a spatial working memory task was only transient, which they believe accounted for the deficits observed in PrL‐lesioned rats in other labs. In their study, when rats had to retain spatial memory during a 5 or 30 minute delay, PrL‐lesioned animals were only impaired when a distractor task or stimulus was introduced during the interval. Distractions could account for deficits observed in studies that observed a greater‐than‐transient effect of PrL damage on working memory, for example in a study by Seamans et al. (1995), during which rats were removed from the environment for lidocaine injections. The role of PrL in working memory is therefore unclear, and could be explained by some 96 increased inertia in establishing or reinstating a behavioral or environmental “map” when an animal is first introduced, or reintroduced, into a behavioral context. The interpretation that apparent working memory deficits can be accounted for by a decrease in the ability of PrL rats to switch to the appropriate behavioral context is not the only candidate explanation, but it appears to fit with data demonstrating that PrL lesions also impair rats’ ability to switch to a new behavioral set (Ragozzino et al., 1999a; Ragozzino et al., 1999b; Birrell and Brown, 2000; Ragozzino et al., 2003). These studies have primarily used two types of tasks. In one, rats running on a “T” or “Y” maze are trained to use a response‐strategy (that is, to always use the same action; e.g., a right turn, or to follow a visual cue); a “task switch” takes place when the experimenter begins to reward the rat only when it goes to a certain place. PrL rats take more trials to switch from using a response strategy than control, intact rats. Another task that has been used was created by Birrell and Brown (2000). In this, rats learn to dig either in wells with particular textures, or in wells that have a particular odor. An intradimensional shift takes place when the type of texture or odor the rats are expected to orient to is switched. On the other hand, an extradimensional shift takes place when rats are no longer rewarded for consistently following any odor, but instead for digging in the correct texture. Rats with PrL damage are differentially impaired at this latter task. 97 On the surface, these data appear to be consistent with the hypothesis that PrL is like dorsal lateral prefrontal cortex, at least with respect to attentional set‐shifting. It turns out, however, that in at least one version of a switch task, the response‐to‐place switch, PrL rats are impaired not in the acquisition of the switch, but in the next‐day retention of the switch (Rich and Shapiro, 2007). Moreover, these deficits only appear the first few times the rat is required to switch the strategy it uses; after repeated switches, vmPFC‐lesioned rats resemble controls. These data are consistent with the difficulties rats have when re‐introduced to a behavioral context, as described by Gisquet‐Verrier and Delatour. In other words, even without PrL, a rat can switch between attentional/behavioral sets without problem, particularly if the rat has previously encountered the situation in which a switch is needed. How can these data be explained? Although there are a number of possibilities, two seem particularly parsimonious. The first is that the vmPFC is involved in retrieving the behavioral context of an environment when re‐introduced to a familiar setting (i.e., “What do I need to do here?”), possibly by the cycling of signals through mPFC‐temporal lobe loops. This possibility is consistent with the observation that damage to the mPFC causes a decrease in the stability of the hippocampal place‐map (i.e., the tiling of neuron firing fields in the environment, considered to be a representation of the environmental and behavioral context), specifically following introduction of a novel condition or stimulus (Kyd and Bilkey, 2003, 2005; note, however, that these lesions were primarily of the dorsal mPFC). This explanation also may speak to data suggesting that the vmPFC 98 contributes to the “feeling of knowing” during exposure to a familiar stimulus (Pannu et al., 2005; Schnyer et al., 2004; Fellows and Modirrousta, 2008b). A second explanation fits with a hypothesized involvement of the vmPFC in stabilizing emotion and behavior when the environment is perceived as predictable or controllable. That is, high levels of confusion, such as during the first exposure to an extradimensional shift or following distraction during a working memory task, may evoke increased stress or emotional responses. There is evidence to suggest that the vmPFC is part of a negative feedback loop to reduce this response, as elaborated on in the next section. What does the ventral mPFC do for behavior, cognition, and emotion? Examining the expectation of reward and punishment The extensive connectivity of IL and ventral PrL with brainstem, hypothalamic, and amygdala systems is a clear indication that the region is involved in driving, coordinating, and regulating emotional responses (see Chapter 2). Among the first studies dissociating the rodent dorsal and ventral mPFC in emotional responsiveness was a pair of experiments performed by Morgan and LeDoux (Morgan et al., 1993; Morgan and LeDoux, 1995), who found that damage to the vmPFC (IL and ventral PrL) impaired extinction to a conditioned, fearful stimulus, but that damage to the dmPFC caused an overall increase in reactivity to the conditioned stimulus during training and extinction. This story was extended by Milad and Quirk (2002), who observed an 99 increase in the number of conditioned‐stimulus‐sensitive neurons in IL the day following extinction training as compared to the day of conditioning and extinction training; moreover, the response of these neurons was correlated with the degree to which the rats learned to extinguish the behavior. More recently, details have been added to this story, including that fear extinction requires plastic changes in the dendrites of intercalated neurons (amygdala inhibitory neurons projecting feed‐forward inhibition to the central amygdala, see Chapter 2), and that these plastic changes depend on an intact IL (Amano et al., 2010). The role of IL in extinction learning suggests that the region will represent a stimulus (a CS) under conditions in which that stimulus has emotional meaning, and that some comparison process takes place between the coded meaning and the experienced outcome. It would be hypothesized, therefore, that the region represents the expectation of the outcome following a CS, so that feedback about the outcome can be matched to that expectation. This same principle of expectation was just discussed in the context of the involvement of the dmPFC in representing relative action value. Also, the IL is not unique in receiving input from temporal lobe regions, such as the perirhinal cortex and amygdala, about stimuli and their emotional meaning. The orbitofrontal cortex (OFC) is also thought to be critical for representing the values of stimuli, which is directly reflected in tasks requiring rats to dynamically change their approach behavior to specific stimuli (Schoenbaum et al., 2002; Chudasama and Robbins, 2003; Boulougouris et al., 2007; Young and Shapiro, 2009). The vmPFC is not involved in 100 stimulus‐value reversals of this type (Chudasama and Robbins, 2003; Chudasama et al., 2003). Stimulus‐emotion associations of the IL must therefore be distinct from the stimulus value associations of the OFC. Experiments in the lab of Steven Maier have linked the emotional control by IL with the IL’s representation of expectation, by examining the IL‐dorsal raphe connection during exposure to an escapable as compared to inescapable stressor (Amat et al., 2005). Rats were exposed either to an escapable or inescapable stressor by placing them in a box and shocking their tail until, in the case of the escapable condition, the rat terminated the shock by turning a wheel. Rats in both escapable and inescapable conditions were examined either with or without IL inactivation. The results suggested that in the escapable stress condition, the serotonergic system was less activated as compared to in the inescapable stress condition. Moreover, this reduced activation of the dorsal raphe during escapable stress depended on the IL, even though rats without an IL were just as capable of learning how to escape the stressor. Further work found that the escapability of the stressor may not have been the important factor: if the stressor and its termination were predictable to the rat, the same results were observed (Maier, personal communication). The implications of these studies are that expectation‐related activity of the vmPFC is not required to learn actions to escape a stressor, but is required to temper emotional response to a stressor that can be predicted and controlled. 101 This theoretical outline of the rodent vmPFC as a site that regulates emotion based on value expectation is also consistent with observations made in the human. Bechara et al., (1999), for example, found that patients with damage to the vmPFC did not generate anticipatory emotional responses (measured by the resistance of their skin, the SCR, which changes when a person sweats), but were capable of generating reactive emotional responses. These patients consequentially expressed different behavior than controls in a gambling task, in which they were less likely to change their choice behavior after receiving large negative outcomes from a high‐risk deck of cards. One can imagine that if the expectation were a false one, for example, the expectation that a placebo medicine might reduce an individual’s suffering, the vmPFC would become engaged. In fact, the placebo effect has been associated with activity in the human vmPFC (Zubieta and Stohler, 2009; Qiu et al., 2009). Forming outcome expectation from history: hypothesized role of the hippocampus in creating mPFC expectation Since Scoville and Milner’s 1957 report of anterograde amnesia in a patient with bilateral damage to the medial temporal lobe, the hippocampus has been the target of study for memory formation in the brain. 10 In the rodent literature, most paradigms 10 Interestingly, one of the largest breakthroughs in this field can be credited to researchers who stepped away from linking the hippocampus to memory, and instead linked the hippocampus to spatial cognition (O’Keefe and Nadel, 1978). Our current understanding of memory formation in the brain could not have 102 that demonstrate a hippocampal dependence on memory‐formation fall into one of several categories. There are spatial tasks, in which rodents must learn that a given area of space is associated with a rewarding or aversive outcome (e.g., Morris et al.,1982). There are context‐learning tasks, in which rodents must learn the association between a spatial environment and, most commonly, a punisher (e.g., Kim and Fanselow, 1992; Philips and LeDoux, 1992; Anagnostaras et al., 1999). And finally there are trace‐tasks, in which the rodent must learn the association between a conditioned stimulus and an outcome, with the two separated by a temporal interval (rabbit: Solomon et al., 1986; Moyer et al., 1990; rat: MEchron et al., 1998; Weiss et al., 1998; Takehara et al., 2003). Evidence suggests that performance of these initially hippocampal‐dependent tasks become independent of the hippocampus over time (Squire, 1992). Although certain spatial tasks appear to depend on the hippocampus even when the environment is well‐learned and the memory is remote (Riedel et al., 1999; Teixeira et al., 2006), rats that have been reared in an specific environment do not exhibit memory or navigation deficits in the environment following hippocampal lesions (Winocur et al., 2005). 11 Associations between environmental stimuli and emotion become independent of the hippocampus over time (Kim and Fanselow, 1992; Philips and LeDoux, 1992; been possible without decades of research that ignored memory formation and focused on how the hippocampus is involved in spatial cognition. 11 Although these hippocampal rats are unimpaired at navigating to reward locations in familiar environments, they do not appear to be capable of using the space flexibly, for example when a novel barrier is introduced to the environment. However, it could also be argued that this manipulation changes the space, and therefore requires systems devoted to the formation of new spatial memories; i.e., the hippocampus. 103 Anagnostaras et al., 1999). Finally, trace memories become independent of the hippocampus (Kim et al., 1995; Takehara et al., 2003; Quinn et al., 2008). Importantly, although memory expression in specific behavioral paradigms becomes independent of the hippocampus, there is a fair degree of evidence to suggest that the hippocampus is always necessary for contextual or episodic “richness” of the memories (Nadel and Moscovitch, 1998; Winocur et al., 2007). Thus, in the absence of a hippocampus, the behavioral expression of a memory may be based on an entirely different physiological and cognitive substrate; that is, the association between the stimuli and the behavior in the absence of the hippocampus is built from different physiological (and psychological) machinery. This does not, however, change the fact that some additional machinery is built to support an long‐term association. Even more interesting, and which will be presently addressed, is that this additional machinery is not merely redundant with hippocampal function, but is required to efficiently recover older associations (after which, presumably, the hippocampus “reconsolidates” the older memory, likely adding to it the new contextual details). Recent experimental results suggest that the mPFC supports the expression of remote associations that initially depended on the hippocampus (Takehara et al., 2003; Frankland et al., 2004; Maviel et al., 2004; Teixeira et al., 2006; Quinn et al., 2008). The expression of spatial, contextual, and trace memories are disrupted following mPFC disruption if the memory was learned more than a few weeks prior to testing. Moreover, the participation of the mPFC in remote memory expression depends on 104 prefrontal NMDA receptors, the glutamatergic channels responsible for long‐term associative plasticity, during the first few two weeks after learning (but not the second two weeks; Takehara‐Nishiuchi et al., 2006). Takehara‐Nishiuchi and McNaughton (2008) also observed an increase in trace‐interval activity, presumably reflecting expectation, following overlearning or consolidation of a trace‐eyeblink memory task. Together, the results suggest that during consolidation, the hippocampus mediates a process that shifts the mPFC from being irrelevant to a memory, to becoming important for that memory, and this correlates with putative expectation‐related activity of the neurons. The expectation‐related activity may either directly stimulate sensory and motor systems involved in memory expression, or might feedback to deep layers of the entorhinal cortex to reinstate the memory traces. The relationship between the hippocampal‐mPFC connection and memory can also be observed in what is called “metamemory,” or the ability of individuals to monitor memory strength for decisions or information seeking. Metamemory in humans can be expressed as reports of “feeling of knowing,” the sense that certain memory information is available, without that information necessarily being recalled; the vmPFC is involved in this sensation (Achnyer et al., 2004; Schnyer et al., 2004; Pannu and Kaszniak, 2005; Chua, 2006; Modirrousta and Fellows, 2008). Although some experiments claim to test metamemory in animals, there has been a fair amount of controversy over how the results are interpreted (Shettleworth, 2010). Although the issue is not taken up presently, one avenue for future investigation may be to 105 understand the role of vmPFC expectation responses in metamemory, and how these are driven or signaled by the hippocampus. The hippocampal‐prefrontal loop, particularly the tight connection between vmPFC and the hippocampal formation, has been implicated in functions beyond memory recall and retrieval. Ventral hippocampus, like vmPFC, has been associated with regulating the stress response during or following a traumatic experience (Henke, 1990; Jacobson and Sapolsky, 1991; Herman and Cullinan, 1997), and is involved in regulating the rhythmnicity of stress hormones. However, the hippocampus does not appear to affect baseline (“average”) stress levels (Tuvnes et al., 2003). The precise manner by which the hippocampus influences the vmPFC during stress reduction, as, for example, in the learned helplessness paradigm used by Maier and colleagues discussed in the last section, has not been experimentally determined. One hypothesis is that expectation‐related activity also plays a role in stress reduction, as discussed above, and that the hippocampus drives this activity as during trace‐eyeblink learning or contextual fear learning. Training expectation by prediction error: hypothesized role of dopamine Although the hippocampus is involved in initially connecting apparently disparate elements, such as a CS and US separated by a trace interval, the expectation of one element from the other must also be capable of adapting according to changes in 106 their contingencies. That is, if the outcome is inconsistent with expectation, then the expectation should change according to the magnitude of this difference (Rescorla and Wagner, 1972). This is called the “delta rule,” and plays a large role in reinforcement learning, the process by which a system learns based on feedback about whether the system’s output matches a desired output. Converging evidence implicates the dopamine system as being a value‐based, prediction‐error signal that provides feedback to frontal networks (Schultz, 2002). The dopamine signal may reinforce both stimulus‐ response and action‐outcome systems in dorsal‐lateral versus medial‐ventral striatum respectively, as well as to the dorsal‐posterior and ventral‐anterior regions of frontal cortex to which they are connected. There may be different learning rates, however, either between dorsal and ventral systems, or alternatively between striatal and cortical regions. The different learning rates create a system that is capable of representing both an integrated value over the long‐term (via the dorsolateral striatum) and the context‐sensitive, dynamically‐changing values in the short term (by ventral and medial frontal cortex; Frank and Claus, 2006; Frank et al., 2007). Whether the mechanisms of dopamine’s effect on frontal cortex is through synaptic plasticity changes, or by intrinsic properties of the frontal neuron, is taken‐up in more detail in the following chapter. Chapter summary 107 The prefrontal cortex resides at the top of the Sherrington’s hierarchy of reflex‐ arcs, and likely plays a fundamental role in establishing action sets, strategies, or schemas, that direct the selection of actions and visceral states. Reciprocal connections between the prefrontal cortex and sensory regions also may provide the basis for the selection of perception (attention) and memory, which has been computationally articulated in models of cognitive control. Dorsal anterior cingulate and prelimbic regions of the mPFC appear to largely participate in the selection of actions based on their context‐dependent consequences. In species with “higher‐level” prefrontal regions, such as the dorsolateral prefrontal cortex in primates, it may also be the case that the representation of potential actions can influence the degree to which these other, higher‐level regions contribute to selecting actions and perceptions (known as the conflict monitoring hypothesis). Ventral prelimbic and infralimbic regions of the mPFC appear to largely participate in the selection of emotions or visceral states based on the context‐dependent consequences of environmental stimuli and executed behaviors. An important factor underlying these functions is the ability to expect a given consequence of an action or stimulus, which may be initially generated by the hippocampus, and may also be trained (according to changing reward contingencies) by the dopamine system. The dynamics of circuit‐level processing involved in mPFC function are addressed in Chapter 4. 108 CHAPTER 4: NEURON COMMUNICATION AND MODULATION IN THE MEDIAL PREFRONTAL CORTEX Chapter aims and challenges Since the connections between mPFC and other regions of the brain are normally reciprocal (Chapter 2), computations of the mPFC can be thought of less as a transformation of inputs to outputs, and more as an integration of information encoded by overlapping anatomical loops. In view of the preceding chapter, the integration of sensory environment, actions, memories, body‐state, and emotions must be useful for action selection, memory retrieval, and emotional selection. The present chapter provides an overview of what is known about how this integration takes place. This begins with a review of how cortical circuits are thought to maintain stable activity states, followed by an examination of how modulatory neurotransmitter systems (dopamine, acetylcholine, norepinephrine, serotonin) might influence the state of information processing. Finally, the chapter will review what is known about brain oscillations, with some discussion of the 6‐10 Hz theta oscillation and a more focused look at the 40‐80Hz gamma oscillation. The gamma oscillation may itself be a manifestation of stable communication states between regions, and the degree to which this communication takes place, the amount of gamma, may be affected by the activity of neuromodulatory systems. 109 A brief history of theory: network mechanisms of persistent activity and the roles of inhibition One of the earliest ideas that linked neural network function to psychological function is the concept that persistent activity in a set of mutually‐exciting neurons might temporarily support a memory. Reverberation in a recurrent network was an essential feature of Donald Hebb’s (1949) theory that neural assemblies form the basis for psychological phenomena. The idea was not original to Hebb, however, who cites a 1940 book by Hilgard and Marquis on conditioning and learning, and who in turn base their idea on the observation of Lorente de No (1938) that the brain contains many closed, or “re‐entrant”, circuits. Hebb recognized that persistent activity alone would not be a sufficient substrate for long‐term memory: Hilgard and Marquis go on to point out that such a trace would be quite unstable. A reverberatory activity would be subject to the development of refractory states in the cells of the circuit in which it occurs and external events could readily interrupt it. . . There are memories which are instantaneously established, and as evanescent as they are immediate. In the repetition of digits, for example, an interval of a few seconds is enough to prevent any interference from one series or the next. Also, some memories are both instantaneously established and permanent. To account for the permanence, some structural change seems necessary, but a structural growth presumably would require an appreciable time. If some way can be found of supposing that a reverberatory trace might cooperate with the structural change, and carry the memory until the growth change is made, we should be able to recognize the theoretical value of the trace which is an activity only, without having to ascribe all memory to it. (Hebb, 1948; p62) 110 At around the same time, these ideas were independently encountered by McCulloch and Pitts (1943) in their development of a “neural calculus” that would later form the basis of computational neuroscience. Instead of emphasizing the transience of the reverberatory activity, they chose to emphasize its robustness, as a potential problem for neuron computation. Like Hebb, however, they also made reference to the memory information contained in persistent activity. The treatment of nets which do not satisfy our previous assumption of freedom from circles is very much more difficult than that case. This is largely a consequence of the possibility that activity may be set up in a circuit and continue reverberating around it for an indefinite period of time, so that the realizable Pr may involve reference to past events of an indefinite degree of remoteness. (McCulloch and Pitts, 1943). These early theorists did not appear to recognize the importance of inhibition to re‐entrant circuits (although the idea of inhibition had been incorporated into McCulloch and Pitts’ neural calculus as a logical “NOT” operator). The absence can be explained by the minor fact that inhibitory neurotransmission was not discovered in the central nervous system until the early 1950’s when Eccles, a Rhodes Scholar and 1963 Nobel Prize winner, discovered inhibitory and excitatory post‐synaptic potentials (IPSPs and EPSPs) to disprove his belief that neuron communication is predominantly electrical (Brock et al., 1952a; Brock et al., 1952b; Todman, 2008). Eccles had been a student of Sherrington, who had predicted inhibition in the nervous system years before in The Integrated Action of the Nervous System (Sherrington, 1909). Several years after Eccles’ discovery, Haldan Keffer Hartline reported the discovery that stimulation of one sensory 111 receptor inhibits the activity of adjacent receptors (Hartline and Ratliff, 1954; Hartline et al., 1956; Hartline and Ratliff, 1957). This extended the phenomenon of inhibition to the network level, in the form of what became known as lateral inhibition. The experiments earned Hartline the 1967 Nobel Prize. Inhibition can also be said to be one of the first predictions ever made by a computer‐based model of a neural network. In 1956, a direct test of Hebb’s theory of cell assemblies was made by Rochester et al. (1956). Using one of the first biologically‐ plausible, neural network computer simulations, they found that reverberation of neural circuits was not robust to experimentally‐introduced noise. At this point we conferred with D. O. Hebb and one of his people, P. M. Milner. Milner had been working on a revision of part of Hebb’s theory to introduce more recent neurophysiological data. The essence of Milner’s idea was that inhibitory synapses, as well as excitatory synapses, are needed and that within a cell assembly most synapses are excitatory, while between cell assemblies most synapses are inhibitory. This idea sounded to us like a plausible cure for the troubles in the first model. It made engineering sense (Rochester et al., 1956). Since this time, neural network models have generally included inhibition. In David Marr’s (1969, 1970, 1971) biologically realistic network model of the hippocampus, inhibitory neurons are included “[W]hose function is to keep the number of [cells] that are active roughly constant during both storage and recall” (Marr, 1971). Hopfield (1982) reinterpreted the idea of recurrent networks with global inhibition into the context of statistical physics, demonstrating that a configuration of active, recurrently‐connected neurons will settle into a stable configuration of reverberatory 112 firing as a low‐energy, “attractor” state. The Hopfield network became both a basic premise and inspiration for many later computational models of the brain. Among the implicit advances the Hopfield network made was its way of computationally articulating the dependence of network activity on both the preceding state of the system and on activity of inputs. More recent examinations of persistent activity have emphasized its putative importance for memory maintained over short‐term intervals (working memory), particularly within the prefrontal cortex. As an example, this comes into play in the cognitive control models discussed in Chapter 3, where persistent activity in prefrontal cortex maintains “rule” or “value” traces that bias sensory and motor association cortex toward specific stimuli and actions (Cohen et al., 1996; Braver and Cohen, 2000; Frank and Claus, 2006; Hazy et al., 2006). Detailed models have identified the importance of NMDA receptors in network‐mediated maintenance (Lisman et al., 1998; Wang, 1999) 12 , and have also revealed that the presence of dopamine can increase the stability of the persistent activity state (Durstewitz et al., 2000a; Durstewitz and Seamans, 2002). In a review, Durstewitz et al., (2000b) subdivided classes of persistent activity models into those in which the recurrent excitation is direct compared with cases in which the recurrent excitation takes place through intermediary units, also called “synfire chains.” 12 A high NMDA/AMPA ratio has been proposed to contribute to the role of the prefrontal cortex in working memory. Although high levels of NMDA receptor RNA have been observed in human post‐ mortem tissue (Scherzer et al., 1998); the rodent mPFC does not contain any higher NMDA/AMPA ratio than does the rat visual cortex (Myme et al., 2003). 113 The review also emphasizes the difference between “continuous” and “discrete” attractors. Discrete attractors may more closely resemble the neural assemblies hypothesized by Peter Milner in the excerpt from Rochester et al., (1954) above, in which the ratio of excitatory to inhibitory neuron connectivity is higher within as compared to between assemblies (or attractors); they may be useful when the brain must actively maintain a specific category or item. In continuous attractors, the active assembly can shift continuously—this is represented by Hopfield nets and can also be demonstrated by the 1‐dimensional, circular attractor models of head‐direction system (Skaggs et al., 1995). Complex networks containing excitatory and inhibitory connections normally have another property: they oscillate. Mathematical characterization of oscillations in complex systems dates back to the early twentieth century, when the Lotka‐Volterra equations were developed for describing predator‐prey relationships. Wilson and Cowan (1972) articulated more precisely the conditions in which excitatory‐inhibitory networks would converge to an oscillating “limit cycle,” as compared to when they may instead asymptote to a single state, or spin‐off into a disordered state. Different subsets of inhibitory neurons have since been associated with contributing to various oscillatory rhythms that, as discussed below, are likely fundamental to neural communication and plasticity. 114 To summarize, the idea that networks can maintain activity, and thereby information, over a delay by recurrent activity is as old as any theory and computer models of the brain. Although there are several ways networks can be organized to support persistent activity (continuous attractors, discrete attractors, and synfire chains), inhibition is always critical for the stability of the reverberating system. Complex systems that involve inhibition also have another feature: they tend to oscillate. It will be shown that these oscillations add an additional dimension to the stability of neural activity and communication. Hypothesized roles of cortical loops in persistent mPFC activity The first demonstration of persistent neural activity in the absence of external stimuli was made by Fuster and Alexander (1971). Since that time, examination of delay‐period activity has become an entire field of study (reviewed in Fuster, 2008). Perhaps the majority of experiments in which delay activity has been observed have examined that primate dorsolateral prefrontal cortex where it has been directly linked to temporary, or “working” memory. Delay activity has been found to have specificity for any number of stimulus or behavioral dimensions, including: previously‐encountered stimuli of any modality, upcoming stimuli, upcoming actions that must be performed, egocentric spatial location of prior or upcoming stimuli, “matching” vs. “non‐matching” behavioral rules for how to react to stimuli separated by the delay, motor sequences, 115 quantities, and stimulus categories. Essentially, persistent activity during a delay interval has been found for any “thing” or abstraction that is behaviorally relevant to the animal. Delay activity is not normally observed to be steady, as might be expected by a stable Hopfield‐style attractor‐state or Hebbian reverberatory circuit, but instead to ramp upward as a target time is approached, or downward from the start of the delay interval. Delay activity sometimes appears to initiate when the stimulus is presented, as might be expected by an attractor/reverberatory circuit (Fuster, 1973), although often the neurons active during the delay period are different from those during stimulus presentation or action. One recent report suggests that the neuron population activity present during stimulus presentation degrades rapidly, and is replaced by ramping activity of an orthogonal set of neurons during the delay period (Barak et al., 2010). These observations are inconsistent with the current model of sustained activity during delay periods. Other regions of cortex may also exhibit delay activity, although some studies suggest the delay activity in the prefrontal cortex is unique in that it is not affected by intervening, non‐behaviorally relevant stimuli (e.g., Miller et al., 1996). Several experiments have followed this lead and examined neural activity in the rodent mPFC during intervals between a conditioned stimulus (CS) and unconditioned stimulus (US) as compared to inter‐trial intervals (Sakurai and Sugimoto, 1986; Batuev et al., 1990; Jung et al., 1998; Pratt and Mizumori, 2001; Baeg et al., 2003; Mulder et al., 2003; Cowen and McNaughton, 2007; Takehara‐Nishuichi and McNaughton, 2008). In these studies, the independent variable distinguishes between an interval of high‐ 116 predictability and high‐behavioral relevance between a CS and US as compared to a period of low‐predictability and low‐behavioral relevance between trials. A major difference between rodent and primate paradigms, however, is that primates perform the experiments from within a tightly confined space, sometimes even with heads fastened to a metal bar; rodents are normally not fully restricted in their movements. A close examination of rodent behavior during delay periods was made by Cowen and McNaughton (2007), who found that even when confined to a limited space, rats would shift their body weights differently for different predictive CS’s, and that this shift was better correlated with delay‐period activity than the stimuli themselves. Thus, “persistent activity” could be inherited from proprioceptive sensory receptor activity, rather than generated intrinsically. Importantly, position‐sensitivity in the mPFC was much stronger when the preceding stimuli predicted upcoming reward probabilities, suggesting that the link between somatosensory and frontal circuits is made only under conditions in which preceding sensory information (the cues) has tagged the somatosensory input as meaningful. The results from Cowen and McNaughton presented above suggest that in the rat mPFC, the persistence of mPFC neurons may be driven by persistence of proprioceptive stimulation from sensorimotor cortex. But this persistence is only observed when the rat is maintaining some expectation of outcome, as if the sensorimotor state is serving as a proxy for the maintenance of a secondary set of information. This may be a clue about the function of the overlap in prefrontal loops: it 117 may be possible to substitute activity through one recurrent loop for activity in another recurrent loop. A result that elegantly demonstrates this principle came from a study by Tremblay et al., (2006). Human subjects were required to remember the order of presentation of seven dots on a computer screen during a delay period, so that they could point to the locations in order following the delay period. The authors found that subjects’ eye‐movements systematically rehearsed the sequence during the delay interval, thus maintaining the prefrontal‐sensorimotor loop for pointing by instead maintaining the prefrontal‐sensorimotor loop for looking. When eye movements were controlled by an intervening task during the delay period, subjects were impaired at subsequently pointing to the locations the dots had appeared. This idea that the mPFC uses information about body state has also been a long‐standing view of Antonio Damasio (1996), whose “somatic marker hypothesis” posits that feedback of the visceral state acts on the ventral divisions of the mPFC to guide decisions. In vivo delay activity may make use of loops with other regions of cortex, and may even, at times, be inherited from bodily feedback; however, in vitro studies suggest that the models of local network should not be completely abandoned. Neurons in brain slices of the ferret prefrontal cortex and of the occipital cortex show sustained activity lasting several seconds, driven by excitatory neurotransmission in the local network and presumably stabilized by local inhibition (Shu et al., 2003). Consistent with the local attractor model, during sustained activity neurons maintained a steady, higher degree of depolarization caused by local network activity (what the authors call an “UP” 118 state), and this state was bi‐stable with a lower level of depolarization when the neuron was less active (what the authors refer to as a “DOWN” state). These results indicate that the local networks by themselves, in both frontal and posterior regions of cortex, are capable of maintaining sustained activity. It is not evidence against the possibility that loops between cortical regions, including ongoing sensorimotor information, contribute to delay activity. In summary, representations in the prefrontal cortex about upcoming or past events are likely to make use of recurrent connectivity in local networks, as well as concurrent activity coming from specific sensory stimulation. The reliance of delay‐ activity on circuits other than just those that are intrinsic to the network may contribute to the observation that selective, “sustained” activity in fact is different from activity during stimulus presentations, and often ramps upward or downward during delay intervals. The data on mPFC persistent activity elicits even more questions. For example: is the bistability of the UP and DOWN states a chronic feature of cortical neurons, or does it depend on the current state of the network? Evidence suggests that neuromodulatory loops, namely the cholinergic and dopaminergic systems, play a role in the degree to which the mPFC is responsive to new stimuli, discussed in the next section. 119 The role of neuromodulatory loops for mPFC circuit function Neuromodulatory transmitters (such as acetylcholine, dopamine, norepinephrine, and serotonin) play an important role in determining the excitability of neurons, the plasticity of their synapses, the sets of inputs the neurons respond to, and in generally determining the firing behavior of the neurons. Much work that has examined how neuromodulators affect cellular, synaptic and network behavior has been spearheaded by Eve Marder, who examined a small network of neurons in the stomatogastic ganglion of lobsters that control gut movements (Marder and Thirumalai, 2002). The present review examines several examples that have helped develop an undersatanding of the neuromodulation of the mammalian cortex. Some insight into the network state changes that can be caused by neuromodulators in the cortex comes from work by Michael Hasselmo and James Bower (1993), who modeled the effects of acetylcholine on cortical circuits. The theoretical work suggested that acetylcholine suppresses postsynaptic responses to intrinsic, excitatory synapses, while increasing post‐synaptic responses and synaptic plasticity of synapses made by afferents from other regions onto the neurons. In this model, the presence of acetylcholine determines the degree to which the network is engaged in encoding as compared to retrieval. Later work by Hasselmo (e.g., Hasselmo et al., 2002a, 2002b) theorized that encoding and retrieval processes in the hippocampus may also fluctuate within cycles of the 6‐10 Hz theta rhythm, an oscillation dependent on 120 input from the cholinergic, basal forebrain nuclei. This prediction has been partially confirmed by recent data showing that hippocampal CA1 region is influenced by CA3 and entorhinal cortex on different theta phases (Colgin et al., 2009). In addition to a link between acetylcholine and the theta oscillation, computational models also suggest a link between acetylcholine and the 30‐80 Hz gamma oscillation (Borgers et al., 2005), discussed in the next section. Acetylcholine acts on postsynaptic muscarinic receptors found on cortical and hippocampal pyramidal neurons, many of which reduce dendritic shaft potassium conductance and effectively increase signal to noise. Nicotinic receptors on the terminals of thalamic inputs into the mPFC may also facilitate the transmission of input signals (Gioanni et al., 1999; Lambe et al., 2003; although synaptic efficacy of the intrinsic circuitry may also be transiently increased). Although many of the details for how acetylcholine and other neuromodulators act on cortical circuitry still must be worked‐out, the Hasselmo and Bower model provides a framework for thinking about all neuromodulatory neurotransmitters. The idea is that specific neuromodulators have the power to shift the set of neuroanatomical loops that cortical neurons are engaging in at a given time—for example, acetylcholine may shift the balance between the engagement of local‐circuit loops compared with inter‐cortical loops. Another neurotransmitter that might be hypothesized to fit into this idea is norepinephrine. Norepinephrine is minimal or completely absent from the forebrain during REM sleep, when the brain enters a waking‐like, but hallucinatory state (e.g., Lena et al., 2005; Mallick et al., 2002). On the 121 other hand, high levels of stress, associated with increased norepinephrine, can stifle creative association (e.g., Beversdorf et al., 1999; see Heilman, 2005). It is possible that neorepinephrine, like acetylcholine (though acting in a different, still unspecified way), increases the degree to which the cortex responds to external stimuli as compared with intrinsic circuitry. The effects of dopamine should also be considered within this framework. Many hypotheses of dopamine’s network function are derived from the observation, discussed in Chapter 3, that dopamine neurons seem to encode a value‐based, prediction‐error signal. Such a prediction error signal could be valuable in reinforcing action sets (including attentional sets) that had been established prior to the stimulus that elicited the dopamine signal. This could lead to the persistence of action sets even in the presence of distraction. In this case, dopamine may help stabilize the established activity state. Paradoxically, a positive prediction error signal could also mean that a new set of stimuli, associated with a new action set, is more advantageous than the currently‐held one, in which case the currently activity state should be “bumped” to a new one (also called “gating”; Braver and Cohen, 2000). Empirical and theoretical work suggests that at least one of several factors could explain the apparently contradictory effects dopamine is thought to have on prefrontal circuits. First, dopamine neurons have a tonic, baseline firing rate, yet also exhibit burst‐pattern firing, and changes in either may affect activity in the prefrontal network in different ways. Braver and Cohen (2001) have proposed that fluctuations in tonic dopamine can regulate the stability of 122 the network activity state, while phasic bursts of dopamine might cause a shift to a new activity state. Second, dopamine acts not only directly on prefrontal neurons, but also on the striatum of the basal ganglia. A number of recent models by Michael Frank and Randy O’Reilly, with accompanying data from Parkinson’s patients, suggest that dopamine‐determined activity of striatal D1 versus D2 receptor neurons regulates the currently‐active set of prefrontal neurons, via further signaling through the pallidum and thalamus (Hazy et al., 2006; O’Reilly and Frank, 2006; Frank and Claus, 2006). Third, as is also the case with respect to cholinergic input, dopamine signals do not act alone, but in conjunction with GABAergic, and to some degree glutamatergic, signals originating from the same source nuclei. In fact, stimulation of the VTA results in a surprisingly large glutamatergic signal in prefrontal cortex at monosynaptic latencies (Lavin and Seamans, 2005) There is some empirical evidence that increased levels of dopamine can increase the stability of the prefrontal activity state. Generally, dopamine increases the excitability of prefrontal neurons (Henze et al., 2000), but this effect is complex. Dopamine acts by enhancing persistent sodium currents, suppressing slowly‐inactivating voltage‐sensitive potassium channels, differentially affecting a variety of calcium channels, enhancing NMDA and GABA‐A receptor currents, and reducing the effect of AMPA receptors (see Durstewitz and Seamans, 2002). A computational model that integrated these effects found that the combined action of dopamine on a cortical neuron might be to strengthen the degree to which that neuron is settled into a high‐ 123 firing rate, “activated” state as compared to a low‐firing, “deactivated” state. This finding is also based on the prior observation that the biophysical properties of NMDA receptors allow neurons in an intrinsically‐connected network to enter a stable, high‐ firing rate state (Lisman et al., 1998; Wang, 1999). In other words, dopamine may increase the activation energy separating a low‐firing rate state from a high‐firing rate state. Data at odds with the idea that dopamine stabilizes the firing rates of prefrontal neurons includes observations by Williams and Goldman‐Rakic (1995), who found that dopamine antagonists directly applied to a neuron tended to stabilize delay‐period activity. Further complexity can be added to this story when, instead of directly applying dopaminergic agonists and antagonists, the VTA is stimulated. Intracellular recordings in anesthetized rats reveal that neurons can be divided into those spontaneously fluctuating between a more depolarized, “UP” state and a more hyperpolarized, “DOWN” state, and those that do not exhibit this fluctuation. Spontaneous activity of the non‐fluctuating neurons is inhibited with VTA stimulation, while fluctuating neurons were not (Lavin and Seamans, 2005). At the same time, neurons appeared to be more excitable during the minutes following VTA stimulation, which on the surface appears to fit better with dopamine providing a gating signal, rather than increasing firing stability. A separate study found that VTA stimulation reduced the efficacy of amygdala stimulation on vmPFC neurons (Floresco and Tse, 2007), which could be thought of as an increase in the robustness of the activity state 124 against outside influences. Unfortunately, due to the effects of anesthesia on neural activity, it is difficult to fully interpret these data without similar studies being performed in the waking animal, and investigation of the contribution of regions such as the basal ganglia. One other way that dopamine may affect network activity is by increasing communication between specific regions of the brain. A recent report found that the coherence between the theta oscillation in the ventral hippocampus and PrL increases when a rat reaches a choice point of a maze (Benchename et al., 2010). In this study, the coherence further increased when the rat learns which direction to go at the choice point, which may correspond with an increase in phasic dopamine at that portion of the maze. Confirming this possibility, the authors found that local delivery of a dopamine agonist in the anesthetized rat also increased hippocampal‐PL coherence. Still missing from this list of neuromodulatory neurotransmitters is serotonin. The importance of the serotonin system is profound: the most widely prescribed drugs in the United States are those that act on the serotonergic system. Most of these are used for the treatment of depression, though manipulations of the serotonergic system also improve obsessive compulsive symptoms (Winslow and Insel, 1990), and the commonly used recreational drug MDMA specifically affects serotonin release. Although a framework for thinking about serotonin’s effects on cortical networks is still missing, a sense of its effects can be gained from a small sample of electrophysiological 125 and behavioral investigations. Serotonin acts on the thalamus by inhibiting neurons, both directly and by acting on local inhibitory neurons (Monckton and McCormick, 2002). In the mPFC, serotonin manipulations appear to influence impulsive behavior. Serotonin agonists in the mPFC increase impulsive responding and antagonists decrease impulsive responding, suggesting that serotonin may inhibit IL (reviewed by Robbins, 2005). The array of different seroronin receptors, and differential activities between cortical regions, suggests that serotonin’s actions are likely to be more complicated than providing an inhibition signal. In summary, modulatory neurotransmitters have important roles in determining the activity state of prefrontal networks, but the specific physiological effects and computational consequences are only partially understood. Acetylcholine and norepinephrine have been proposed to increase the degree to which the networks respond to current sensory activity, and their reductions may increase the degree to which neurons within a network are driven by intrinsic connections, such as during memory retrieval (previously‐established synaptic connections, if the Hasselmo and Bower model is correct, these synapses may be weighted relatively higher with decreased acetylcholine) or with creative insight (the generation of novel synaptic connections, possibly increasing with decreased norepinephrine). Cholinergic input into the hippocampus is also associated with a 6‐10 Hz hippocampal theta rhythm which may provide windows in which the network can respond to intrinsic (CA3) versus afferent (entorhinal) inputs. Dopamine may influence cortical circuit activity differently 126 depending on the the dynamics of its tonic versus phasic release, its actions directly on the cortex as compared to indirectly through the striatum, and the degree to which non‐ dopaminergic signals originating from the same nuclei are also participating in the modulation. Computational work suggests that dopamine acts on frontal circuits to stabilize previously‐established network activity, although empirical results confirming this theory have suggested that this story is more complex. Finally, serotonin appears to provide inhibitory signals to both thalamus and vmPFC, its actual effect on cortical processing remains to be determined. The effects of neuromodulation on the state and function of cortical networks is fundamental to understanding how the networks work as a whole, and will prove to be an important factor in interprettting neural activity differences between different cognitive and behavioral states. Oscillations and cortical communication: focus on gamma Oscillations are so ubiquitous in nature, that many cultures and philosophers have speculated that the rule is unfaltering, and that cosmological history itself is an oscillation. 13 The observation that there are electrical oscillations in the brain can be 13 This is often referred to as eternal recurrence, or eternal return. The Mayan calendar was cyclical, and ideas of eternal return have been found in the writings of Greek philosophers Empedocles and Zeno of Citium. This concept was most notably dealt with by Friedrich Nietzche, reviewed also by a short essay by Borges (1941). Isaac Asimov (1956) plays with the idea in his short story “The Last Question.” Even scientific cosmology included models of a cyclical universe; e.g., Stephen Hawking (1988) proposed alternating cycles of “big bang” and “big crunch,” although data now suggests that the universe will continue to expand as entropy asymptotes (i.e., until the end of time). 127 traced at least as far back as the work of Herr Doktor Hans Berger, who entered the field interested in how electrical scalp signals might be transmitted through the air as a mechanism for telepathy (Buzsaki, 2006). Oscillations of neuron firing can be caused by generators intrinsic to the neuron, or by network interactions. Intracellular generators include circadian neurons in the suprachiasmatic nucleus (e.g., Welsh et al., 2010) and subthreshold 3‐7Hz activity of entorhinal stellate cells (Alonso and Llinas, 1989). Even when neurons undergo intrinsic oscillations, network interactions are still involved in setting and synchronizing the oscillation, and sometimes determining the oscillation speed. The behavior of synchronized, oscillatory neural firing is normally observed by recording the local field potential. The local field potential reflects the sum of nearby electrical sources and sinks caused by membrane depolarization, and can be partially predicted by on local neuron firing (Rasch et al., 2009; Eeckman and Freeman, 1990; Berens et al., 2009). A number of overlapping oscillations can be observed in the cortical local field potential depending on the behavioral state of the animal and activity state of the recorded network (reviewed by Buzsaki, 2006) Among the more prominent oscillations observed in local field potentials is the 30‐80 Hz gamma oscillation. LFP oscillations around 40Hz have long been reported in a range of species, including catfish, toad, frog, turtle, rabbit, rat, hedgehog, dog, cat, monkey, chimpanzee, and human (reviewed by Sheer and Grandstaff, 1970). Gamma normally increases in strength when local neuronal activity increases by the stimulation of afferents, such as in the primary visual cortex during visual stimulation (e.g., Eckhorn 128 et al., 1988; Gray and Singer, 1989a; Young et al., 1992; Berens et al., 2008) or in the olfactory system during odor stimulation (e.g., Sheer and Grandstaff, 1970; Bressler and Freeman, 1980). The increase in power reflects an increase in the coherence of local synaptic activity. Investigations of the factors which determine the frequency, power, and between‐region coherence of gamma have proven to be an informative approach to understanding neural communication. The broad band of frequencies over which gamma is defined, 30‐80 Hz, reveals the variance of the gamma oscillation frequency, and the difficulty investigators have had pinning down a precise number. There has also been recent evidence of a “fast gamma”, roughly 80‐200Hz, with patterns that differ from 30‐80Hz gamma (e.g., Sirota et al., 2008; Colgin et al., 2009). Some variation has been observed in the peak frequency of gamma across species; for example, the cat olfactory bulb and piriform cortex exhibit gamma around 40Hz, while the olfactory bulb and piriform cortex of rabbits and rats appear to center above 50 Hz (Bressler and Freeman, 1980). It is possible that observed species differences are related to other experimental confounds, such as how animals are stimulated during recording. Different reports have been made about gamma frequencies within spieces. For example, in a set of experiments performed in the lab of Gyorgy Buzsaki, spontaneously‐exploring Sprague‐Dawleys rats exhibited a mean, hippocampal‐gamma frequency of 53 Hz (Csicsvari et al., 2003). In a report from the Moser lab, however, hippocampal gamma in foraging, Long‐Evans rats was reported to center just above 40 Hz (Colgin et al., 2009). Another report from the 129 Buzsaki lab found gamma in the rat mPFC to also peak between 50‐60 Hz (Sirota et al., 2008; see also Chapter 7). Gamma frequencies between 50‐60 Hz are consistent with those observed in primate neocortex (e.g., Womelsdorf et al., 2007; Berens et al., 2008, 2009). One older study in rabbits induced hypothermia and measured stimulus‐evoked gamma frequency in the olfactory bulb as the animals warmed themselves (and, in two rabbits, as they froze to death; Putkonen and Sarajas, 1968). The authors observed a linear relationship between body temperature and gamma frequency—although gamma frequency at normal body temperatures still ranged across at least ten degrees. These data are consistent with temperature‐frequency relationships observed in non‐ mammals (reviewed in Bressler and Freeman, 1980). Other physiological patterns, including spontaneous firing rate and the size of an evoked population spike, have also been found to be affected by brain temperature (Erickson et al., 1996). The specific mechanisms by which temperature might affect gamma frequency have not yet been examined. Gamma is highly dependent on inhibitory interneuron activity, which has been investigated in vitro, in vivo and in neural network models. Isolated networks of inhibitory neurons are capable of generating gamma when stimulated with a metabotropic glutamate receptor agonist (Whittington et al., 1995). So called “interneuron gamma,” or ING, has been extensively modeled (e.g., Traub et al., 1996a; Wang and Buzsaki, 1996; reviewed by Whittington et al., 2000). The models have also suggested that not all inhibitory neurons contribute equally to gamma, and that gamma 130 generation depends on the fast‐spiking (FS), basket/chandelier neurons. By genetically modifying FS, parvalbumin neurons to depolarize with light, Cardin et al. (2009) found that selective stimulation of these neurons at 40Hz‐50Hz was sufficient to evoke network gamma, an effect not observed when excitatory neurons (expressing calcium‐ calmodulin dependent kinase) were stimulated at the same frequencies. Fast‐spiking neurons are typically connected with one‐another via electrical, gap‐junction synapse (Gibson et al., 1999), as if forming a single, functional unit; these connections are necessary for cortical gamma (Draguhn et al., 1998; Deans et al., 2001). The dependence of gamma on inhibitory circuits is consistent with the dependence of gamma frequency on the time constant of inhibitory post‐synaptic potentials; these can be increased, along with corresponding increases in gamma frequency, by application of barbiturates (e.g., Stumpf, 1965; Whittington et al., 1995) Although gamma can be modeled using only inhibitory neurons, electrophysiological recordings from behaving animals suggest that pyramidal neuron firing often drives these fast‐spiking neurons to fire (Csicsvari et al., 2003; see also Chapters 8 and 9). Thus, pyramidal neurons can participate in determining the gamma frequency. One confirmation of the influence of excitation on gamma frequency is the observation that activation of NMDA channels specifically on inhibitory neurons increases gamma frequency (Mann and Mody, 2010). The role of excitatory neurons has also been incorporated into many models (e.g., Traub et al., 1997; Brunel and Wang, 2003). Some models require synchronized excitatory neuron activity for gamma 131 generation, referred to as pyramidal‐interneuron gamma (PING, reviewed by Tiesinga and Sejnowski, 2009; Whittington et al., 2000). The dominance of “ING” or “PING” mechanisms of gamma generation may depend on the experimental preparation that is used (Bartos et al., 2007). An interesting confirmation of the interactions between excitation and inhibition in determining gamma power and frequency comes from close inspection of the patterns of individual gamma cycles. Individual cycles tend to be delayed, that is, the instantaneous frequency is reduced, immediately following larger (higher power) gamma cycles as compared to low‐power cycles (Atallah and Scanziani, 2009). The data suggest that higher excitatory drive in high‐amplitude cycles may causes increases in the level of inhibition, causing neuron membrane potentials to take longer to return to baseline. As noted above, gamma synchronization in a region is known to increase with increased activity of its afferents. Increased input to a region drives both inhibitory and excitatory neurons, resulting in greater gamma‐rhythm synchrony (Kopell and LeMasson, 1994). It is also thought that cholinergic stimulation can increase gamma‐ rhythm synchronization (Borgers et al., 2005). A number of studies have found relationships between the timing between brief bursts of gamma and the phase of the theta oscillation (in the hippocampus: Bragin et al., 1995; Colgin et al., 2009; in the mPFC: Sirota et al., 2009). One of these studies, Colgin et al. (2009), was able to directly relate 40 Hz gamma bursts to periods of influence from hippocampal CA3 onto CA1, as contrasted with high‐frequency events, roughly 80‐120Hz, which appeared to reflect 132 influence of entorhinal input. A similar distinction between gamma (40‐70Hz) and high‐ frequency (120‐160 Hz) events at distinct phases of theta has been observed in the mPFC (Sirota et al., 2009). In the entorhinal cortex, two distinct gamma oscillations can be observed (Chrobak et al., 1998; note: although Figure 1 of this article shows a peak around 55Hz, the peak appears not to be explicitly examined by the authors). The authors of this report find gamma sinks and sources to be strongest in layer I, with a secondary, slightly offset source/sink in layers II/III. Perhaps the largest breakthrough in the study of gamma oscillations was the finding that gamma in regions involved in the same computation tend to synchronize (Eckhorn et al., 1988; Gray et al., 1989a; Engel et al., 1991a, 1991b; reviewed by Fries, 2009). Although the precise causes of synchronization are unknown, it is likely to be an emergent effect from entrainment of two regions caused by reciprocal connectivity (see also Traub et al., 1996; Borgers et al., 2005b). The implications of increased synchrony are profound. Empirical studies (Womelsdorf et al., 2007) and theoretical work (Tiesinga et al., 2001; Tiesinga et al., 2004; Borgers et al., 2005b) suggest that communication takes place between gamma‐synchronized circuits, but not between non‐synchronized circuits. Neurons will not respond to single inputs, only to large sets of inputs. Brief windows of high input activity are generated within the gamma oscillation, during which presynaptic networks can effectively stimulate postsynaptic networks. This window is also determined by the membrane time constant of neurons responding to EPSPs (roughly 16ms for layer V pyramidal neurons: Kim and Connors, 133 1993; Koch et al., 1996), which fits within the gamma cycle. The entrainment of synchrony between two regions may correspond with the exclusion of a third region, thus removing the influence of neurons firing within that set, and thereby increasing the robustness of an attended stimulus or engaged action set. In other words, just as static inhibition is necessary for the stability of a sustained neural activity state, as described above, oscillatory inhibition may also help stabilize the sustained communication between two regions. Since sources and sinks are strongest in superficial layers, it is possible that input arriving from the thalamus is not subject to the same limitations. Neural synchronization within specific oscillations, particularly gamma, is an important means by which the brain controls communication. Gamma is observed across a wide range of species. Its generation involves recurrent connections within populations of fast spiking inhibitory neurons, and their interactions with populations of excitatory neurons. The frequency of gamma is determined both by the time constant of inhibition, as well as the excitatory drive onto inhibitory populations. Gamma power is a reflection of local synchronization, which can increase with increasing input to a region. Synchronization of gamma between areas of cortex also takes place when those regions are engaged in the same computations. Empirical and modeling work has suggested that gamma synchronization provides a mechanism for stabilization of communicating networks, an extension of what is already known about the role of inhibition in the stabilization of network activity. 134 State changes in brain circuits during decision‐making The above description of sustained activity, neuromodulation, and oscillations all say something about the “state” of the neural circuit. During delay activity, neurons within the network appear to be either in the active attractor, the UP state, or outside of it, in a DOWN state. The stability of these states is thought to be modified by modulatory neurotransmission, and the level of communication between two regions is reflected by the coherence and power of the regions’ oscillations. In theory, when an animal makes a decision, it enters a transient state, during which actions and their values are momentarily considered and compared. In the discussion on dopamine modulation, it was noted that theta coherence between the mPFC and ventral hippocampus increased when the rat reached the decision‐point (Benchename et al., 2010)—counterintuitively, this pattern was increased when the rat had stronger confidence about its decision, and may have related to dopaminergic modulation. In the previous chapter, it was also noted that the error related negativity (ERN), which also resembles a conflict‐related negativity (the conflict N2), and a feedback‐related negativity (FRN), may be related to response conflict (Yeung et al., 2004), and may be a signature of increased neural synchrony at theta frequency (e.g., Trujillo et al., 2007). The relationship between theta synchrony increases related to higher‐confidence decisions in rats (inferred from the increased coherence following rule‐learning in one 135 rat study) and in lower‐confidence decisions in humans (inferred from the interpretation that ERN, N2, and FRN are more likely to take place on lower‐confidence decisions) is worth reconciling, as the implications of these patterns are likely to be critical during neural communication related to decision‐making. There has also been some ground gained in developing an understanding of the specific neural activity taking place during decision contemplation. Recent neurophysiological examinations in the lab of David Redish have demonstrated some of its unique properties. During indecision, the brain appears to transiently “look ahead”. That is, neurons that normally fire at later positions in the route or sequence a rat is traversing, fire a limited number of action potentials. This was first observed in the dorsal hippocampus (Johnson and Redish, 2007), where the neural activity appeared to transiently “sweep” forward to a goal location. Accompanying this forward sweep was a behavior known as “vicarious trial and error” (VTE) in which the rat hesitates and moves his head from one side to the other (Muenzinger, 1938). The observation of a transient “look‐ahead” was later also observed in the ventral striatum (van der Meer and Redish, 2009) but not the dorsal striatum (van der Meer et al., 2010). The mechanisms to explain how this phenomenon takes place under the specific conditions that it is needed are still unknown. Chapter summary 136 Network dynamics determine the computations that a network will perform. Recurrent loops in neural connectivity create a complex system that is capable of maintaining stable activity states over periods of time, independent of external input. These stable states can be classified as continuous attractors, discrete attractors, or repeating sequences called synfire chains, and the stability generally requires an inhibitory component. It is not necessarily the case, however, that neural activity that is maintained in the brain during a delay period is supported exclusively by attractor dynamics. A good deal of evidence suggests that animals use sensory‐motor feedback to support memory loads over delay periods, and intrinsic activity during delay periods is not static, but shifts between active networks. Network dynamics can also be regulated by neuromodulatory systems. Impressive work has been done to characterize this regulation in simple invertebrate preparations, although the frameworks for thinking about cortical network modulation are still being developed. Hasselmo and Bower have suggested that acetylcholine plays a role in regulating the degree to which neurons in a network are responsive to intrinsic versus extrinsic inputs. Dopamine may act on the frontal cortex to reinforce stable activity states (i.e., increase intrinsic signaling) while at the same time, phasic increases, possibly within the striatum, might allow the network to be “bumped” from the stable state when the animal has encountered an unexpected reinforcer. 137 Inhibitory signaling helps stabilize network states and also, by generating oscillatory activity, determines the state of neuronal communication. Synchronization of particular oscillations, such as gamma, takes place during communication between cortical regions, and the dynamics by which gamma synchronization is formed and broken is likely an important part of how the brain forms and breaks particular perceptual and motor sets when interacting with its environment. Some work has been done to relate network states to decision‐making during behavioral states of hesitation called vicarious trial and error (VTE). In some regions, a “sweeping‐forward” of the neural activity states has been observed, which may reflect exploration by the system of potential outcomes associated with available actions. A primary goal of this dissertation will be to expand upon these data, which includes identifying the specific network dynamics that take place in the mPFC during decision‐making. 138 CHAPTER 5: THE EFFECTS OF AGE ON THE MEDIAL PREFRONTAL CORTEX Chapter aims and challenges No organ is spared the effects of aging. No tissue in the body is identical between a person’s thirtieth, sixtieth, and ninetieth birthdays. Bone density and muscle tone are reduced, hair and skin are affected, and without any question, the aging process takes its toll on the nervous system. As aging affects the body, so it also affects behavior. The changes to muscles, joints, and bones influence the cost of performing certain actions, and in some circumstances might slow reaction times and influence decision‐making. Age‐related vascular change, such as the appearance of hypertension, can be a major factor affecting brain health and cognitive processing (reviewed in Raz and Rodrigue, 2006). There are also many changes to the endocrine system with age (reviewed by Chahal and Drake, 2007) which can affect cognition through actions on brain hormone receptors. 14 As a rule, not all cellular processes and biological molecules are equally affected by the age of the organism, and this unevenness at the microscopic 14 Among the most studied of these is estrogen, though a number of complicating factors influence whether estrogen, and its replacement, help or harm the nervous system in old age (reviewed by Frick, 2009). 139 scale manifests itself at the macroscopic scale as specific, regional declines in the body and brain, which in turn give rise to selective behavioral and cognitive changes. How does aging affect mPFC physiology to change how the network of neurons computes decisions? Can we find network principles which bridge what is known about age‐dependent cellular changes taking place in the mPFC with what is known about age‐ dependent behavioral changes linked to the mPFC? The choice of examining the mPFC is not random, but stems from the observation that the prefrontal cortex and its cognitive functions are differentially compromised by aging (reviewed in the present chapter). It is therefore hoped that paying particular attention to aging in the prefrontal cortex will help to resolve many of the unknowns regarding the mechanisms of age‐ dependent functional decline. The review will begin with a brief overview of what is known about anatomical changes taking place in the aged mPFC. This will be followed by an outline of cognitive deficits that have been associated with mPFC changes in primates, including some interesting results demonstrating that, depending on the situation, aging is associated with both reductions and increases in the engagement of prefrontal cortex. Finally, there will be a brief overview of investigations (and absence of investigations) on behavioral and physiological declines in the rat that implicate the mPFC. There are several challenges in presenting an overview of this literature. The first is appropriately focusing the review on the mPFC, since many observations and 140 theories of age‐dependent functional declines implicate the human and non‐human primate dorsolateral prefrontal cortex. Even focusing on the mPFC itself can lead to unsettling differences between the human and rat, the former of which we, as a species, are most concerned with, the latter being the model that we, as scientists, are most accustomed to dealing with. A balance will be attempted to cover the changes that take place in both primates and rodents. A second difficulty in a review of this literature is to restrict the examination to “normal” aging. Normal is in quotes because, while it usually refers to the members of a set that fall near the central tendency, in this case it refers to those members that are “optimally‐healthy.” The aging process is often accompanied by pathological changes, such as plaque depositions and tangled neuron processes, or vascular lesions of varying sorts and sizes in axon tracts or gray matter. Some species encounter these pathologies more than others, Alzheimer’s disease, for example, is unique to humans. 15 The guiding issue here is to limit the set of variables that are under investigation: what can we say about age‐dependent changes in the mPFC network as distinct from individual‐specific influences which are not necessarily intrinsic to the neurons themselves. The final challenge is to be sufficiently thorough with the review, given the vastness of the scientific aging literature. No attempt will be made to be comprehensive, but the examples given will provide a framework for specific hypotheses on the effects of aging on mPFC circuitry. 15 The Chilean degu also exhibits the etiology of Alzheimer’s disease, though not enough work has been performed to classify it definitively in the same category (van Groen et al., 2009; Inestrosa et al., 2005). 141 Anatomical and chemical changes of the mPFC with age Naftali Raz (2000) concluded a lengthy review on age‐dependent changes to the human brain with the following summary: “Multiple neuroanatomical, neurochemical, and metabolic indicators converge on the notion that brain aging proceeds in a selective fashion, taking the heaviest toll on the prefrontal cortex and on subcortical monoaminergic nuclei to which it is connected via a dense network of projections”(Raz, 2000). In monkeys, Smith and colleagues (2004) found that the prefrontal cortex is highly susceptible to age‐dependent neuronal loss (but only in Brodmann’s area 8a, and not in the adjacent region 46). Most humans and non‐human primate experiments focus on the dorsolateral prefrontal cortex (age‐related changes to area 46 are reviewed in Luebke et al., 2010), which lacks a well‐defined homologue in rats (see Chapter 2). There are a handful of studies, however, that have found age‐dependent changes to the mPFC. Bergfield et al. (2010) found that age‐dependent reductions of gray matter in the anterior cingulate are among the most consistent correlates of age‐dependent gray matter changes throughout the brain. Within the cingulate, these reductions extended as far posterior as the central sulcus, while changes to the most ventral regions of the mPFC appear not to be correlated with age. Age‐dependent gray matter reductions in the anterior cingulate, but not more ventral regions of the mPFC, have been confirmed by other studies (e.g., Grieve et al., 2005; Vaidya et al., 2007). Vaidya et al. (2007) 142 additionally demonstrated that the reductions are accompanied by reduced blood flow to the region. Only a few studies have specifically examined anatomical changes in the rat mPFC with age. Increased loss of cortical neurons has been observed in PrL of male rats, but not females (Markham et al., 2002). However, aged, memory‐impaired female rats do exhibit reduced dendritic spine density in the mPFC (Wallace et al., 2007). In contrast to observations in humans, more dorsal and posterior regions of cingulate appear to be relatively preserved with age (Markham et al., 2007; Yates et al., 2008). On the other hand, neuron reductions found in vmPFC are no greater than those observed in primary visual cortex (Yates et al, 2008), although the rats in the study were not tested for blindness or other visual deficits that might have an influence neuron vitality. Age has also been found to affect dendritic branching of superficial layer pyramidal neurons of the posterior part of the anterior cingulate cortex (Grill et al., 2002). In spite of the relatively subtle effects aging has on the mPFC anatomy itself, aging is likely to influence mPFC function through changes in neuromodulatory nuclei to which it is connected. Goldman‐Rakic and Brown (1981) examined levels of dopamine, norepinephrine, and serotonin in aged as compared with young adult rhesus macaques and found age‐related reductions in only dopamine and these dopamine reductions were strongest in the prefrontal cortex. A recent study examining dopamine fibers in 143 the rat mPFC observed that aging had an effect on the distribution of dopamine terminals, but not of norepinephrine terminals (Allard et al., 2010). They also observed that these changes were distinct between cognitively impaired and unimpaired rats: aged, cognitively impaired rats exhibited reduced dopaminergic varicosities in superficial layers of cortex, while aged, cognitively unimpaired rats exhibited increased dopaminergic varicosities in deep layers of cortex. Another recent study has found that the alpha‐1 and alpha‐2 norepinephrine receptor binding is decreased in the prefrontal cortex of monkeys, and these changes related to cognitive performance (Moore et al., 2005). In addition to dopamine, aging has also been observed to affect cholinergic innervation of the cortex (Sarter and Bruno, 2004); however, results have been somewhat mixed and complex. Strong et al., (1980) observed strong age‐related reductions in markers of cortical cholinergic activity. An attempt to replicate this finding by Baxter et al. (1999) found age‐dependent reductions of the same markers only in the basal forebrain; in frontal cortex, markers of cholinergic activity actually appeared to increase with age. In this study, lower levels of cholinergic markers were associated with better performance in younger rats, while higher levels were associated with better performance in aged rats. There is some doubt in the field, however, about whether these markers used to evaluate cholinergic activity are valid indicators of cholinergic levels in vivo (Carol Barnes, personal communication). Herzog et al. (2003) found that depolarization‐evoked acetylcholine release in the mPFC was much higher in 144 young rats than in aged. The authors claim that previous results demonstrating no age effect of depolarization‐evoked acetylcholine release resulted from biases introduced by the use of inhibitors of acetylcholineesterase (the enzyme that breaks‐down acetylcholine in the synapse). Another line of evidence suggesting that cholinergic innervation into the mPFC decline with age is the observation that cholinergic boutons on layer V pyramidal neurons of the cortex are highly reduced in older rats (Casu et al., 2002). The authors of this finding observed reductions in the number of all synaptic boutons, but those from axons of cholinergic neurons were exceptionally reduced. Although anatomical changes to the rodent mPFC do not appear to be as dramatic as those changes taking place to the human and monkey dorsolateral prefrontal cortex, the mPFC is also not a region that is spared the effects of aging. In particular, declines in dopamine and acetylcholine innervations of the mPFC may significantly affect age‐dependent functional declines. Behavioral and cognitive changes and frontal function in the aged primate Moscovitch and Winocur (1995) tabulated the many parallels observed between functional deficits in patients with frontal lobe damage and those observed in healthy aging. As with the anatomical investigations described above, the offending region appears to be the dorsolateral prefrontal cortex, and targeted research to this region may overlook specific age‐dependent changes taking place in the mPFC. As will be 145 demonstrated, neuroimaging studies have helped with this, and have revealed in some cases that engagement of the mPFC is increased with age. This is thought to be a mechanism by which aged individuals compensate for functional declines, for example in memory tasks (Cabeza et al., 1997; Davis et al., 2008). Other studies have demonstrated that the magnitude at which a region is engaged during a task is only one part of the story, that the timing of frontal involvement is also informative. The dynamics of prefrontal activation suggest that older subjects are more likely to use “reactive” strategies in cognitively‐demanding tasks, in which top‐down control is engaged transiently when stimulated with relevant cues, in contrast with younger subjects who may use “proactive” strategies, in which use of the prefrontal cortex is sustained throughout task trials (Jimura and Braver, 2010). Tasks in which the mPFC has been examined in relation to age‐dependent cognitive declines include goal‐directed attention (which may be synonymous with the ability to inhibit irrelevant stimuli), cognitive “flexibility” (the ability to switch between attentional sets, schemas, or value‐ associations), and memory. Hasher and Zachs (1988) introduced the idea that age‐related cognitive deficits may reflect an inability to selectively inhibit task‐irrelevant information (or perhaps “reintroduced”; Rabbit ,1968, also brought‐up inhibition as being a primary cognitive deficit in aged adult). Although many earlier studies appear to confirm this hypothesis (see Kramer et al., 1994), more recent reviews of the field have revealed a number of confounding factors that may contribute to the results and their misinterpretation 146 (Kramer and Kray, 2006). For example, in the Stroop task, subjects are presented with the name of a color in text that is either the same color (congruent condition) or a different color (conflict condition). Younger subjects take longer to respond when asked to name the text color during a conflict condition; aged adults may be even differentially slower (Dulaney and Rogers, 1994), and also exhibit more intrusion errors than younger subjects (West, 1999). On the other hand, a meta‐analysis of 20 Stroop studies found that aged adults were no more impaired than could be accounted for by a general, age‐ dependent slowing effect (Verhaeghen and De Meersman, 1998). Further investigation observed that, while this may be true when most trials are congruent, aged adults may exhibit differential impairment in Stroop paradigms in which most trials are incongruent (West and Baylis, 1998). Age‐dependent performance changes in the Stroop task have been linked to mPFC functioning by examination of event related changes in scalp‐recorded electrical potentials (ERPs). The anatomical source of individual ERPs can often be approximated by the spatial distribution of ERP signals across the scalp. The amplitude of a negative‐ going potential that takes place 450ms following a correct response or intrusion error (N450) may be one ERP with a source in the anterior cingulate (Liotti et al., 2000). The N450 has been correlated with the magnitude of decision conflict, furthermore, this has been shown to be attenuated with age (West and Alain, 2000; West, 2004). An error‐ related negative potential (ERN), which is also thought to be generated by the anterior cingulate (Holroyd and Coles, 2002; van Veen and Carter, 2002) has also been found to 147 be reduced in aged adults (Nieuwenhuis et al., 2002; Falkenstein et al., 2001; West, 2004). The ERP data are intriguing when paired with a number of fMRI results showing that activation of frontal regions increases in aged adults (Milham et al., 2002; Langenecker et al., 2004; Zysset et al., 2006; Mathis et al., 2009) including increased conflict‐related responses in the anterior cingulate (Milham et al., 2002). This difference may be a case in which the type of activity investigated with ERP as compared with fMRI part from one‐another. Other methods of probing brain activity, in conjunction with assessments of behavioral performance, have helped to confirm an age‐related deficit in goal‐directed, selective attention. In tasks in which aged adults are asked to remember as compared to ignore certain features of presented stimuli, specific patterns can be observed in the stimulus‐evoked ERPs and fMRI activations in posterior cortex. These patterns are attenuated in aged adults, which accompany performance declines in the following memory task (Gazzaley and D’esposito, 2007). A subsequent study observed a correlation between posterior and frontal regions during remember conditions, although the mPFC specifically did not appear to be involved in this relationship (Gazzaley et al., 2007). A general function of the prefrontal cortex is the ability to flexibly adapt to changing reward contingencies, including the ability to shift the dimension of a stimulus that is being attended to. The ability to perform this switch, at least between visual and 148 category dimensions (the Wisconsin Card Sorting Task assess attentional shifts between numerosity, color, and shape), has been linked to the human dorsolateral prefrontal cortex, as observed in human lesion studies (Milner, 1963;), human imaging studies (Monchi et al., 2001; Nakahara, 2002; Lie et al., 2006); monkey imaging studies (Nakahara, 2002); and monkey lesion studies (Buckley et al., 2009). The mPFC, however, may be involved in processing errors and values during these tasks (e.g., Lie et al., 2006; Buckley et al., 2009). It is possible that it may facilitate attentional switches between sensory modalities, and that increases in activity may improve performance on such switches in aged adults (Townsend et al., 2006). Prefrontal regions of cortex have long been known to be involved in learning and memory, many think of this involvement as a role in “strategic” encoding and retrieval processes. Consistent with this idea, Cabeza (1997) observed a generalized increase in prefrontal activity among aged adults during a memory retrieval task, and this effect was higher in high‐performing subjects (Cabeza, 2002). Although the 1997 studied appeared to reveal a dissociation between aged groups with respect to patterns of activation in ventral versus dorsal regions of mPFC, these patterns appeared to vary between tasks, and the low temporal resolution of positron emition tomography imaging (PET), does not allow scrutiny of this effect. Similar apparently compensatory recruitment of the prefrontal cortex has been observed in other experiments (e.g., Logan et al., 2002; reviewed in Grady and Craik, 2000; Park and Reuter‐Lorenz, 2009). As noted in the beginning of this chapter, more detailed examinations of frontal 149 involvement during cognitive tasks have revealed that the dynamics of frontal engagement changes with age, for instance, becoming more reactive to task stimuli as compared to being engaged in a sustained, “proactive” way (Paxton et al., 2008; Jimura and Braver, 2010). All together, research on human neuropsychology reveals decline of the dorsolateral prefrontal cortex to be strongly implicated in age‐related functional decline, with other regions of prefrontal cortex, apparently including the mPFC, increasing their degree of participation in cognitive tasks as a compensatory measure for high performance. The ability of the prefrontal cortex to compensate for functional declines may be a factor in “cognitive reserve” (Whalley et al., 2004; Stern, 2003). Not all indices of mPFC activity show increases in the elderly: ERPs during conflict or error may decrease in aged adults. These decrements may reflect reductions in neural synchrony, either due to changes intrinsic to the network, or reduced stimulation, glutamatergic or otherwise, from other brain regions. As noted by the prior section, there are important differences between species in the frontal regions that exhibit age‐ dependent declines, these are likely expressed behaviorally, and may provide important insight into how the frontal cortex more generally responds to the aging process. Behavioral changes and frontal function in the aged rodent 150 There is a dearth of experimentation on how aging affects the functions of the rodent mPFC. This is both bad for the present thesis, in that there are very few observations with which to generate a theory of mPFC circuit changes with age, and good, in that experimentation of the aged rodent mPFC will inevitably yield additions to the range of what is known. One notable study looked at the difference in the ability of young adult and aged rats to make extradimensional shifts, an idea borrowed from the observation that aging in humans impairs such shifts in paradigms such as the Wisconsin Card Sorting Task (see previous section). Barense et al. (2002) exposed rats to two wells, each with a particular odor and texture, one of which contained a reward. Young and aged rats were both able to learn the association between stimulus and reward, aged rats were slightly impaired as compared to young when the stimulus‐reward associations were reversed within modalities (odor or texture), aged rats were differentially much more impaired when required to switch the attended‐to association. These same patterns of impairment have been observed in mPFC‐lesioned rats (Birrel and Brown, 2000). When the performance of aged rats was correlated with glutamatergic‐receptor binding across multiple regions of the brain, however, a correlation was found in the ventral anterior cingulate (posterior to the mPFC), but not in the mPFC itself (Nicolle et al., 2003), suggesting that regions other than the mPFC itself could have accounted for the differential impairment. 151 The dorsal mPFC is involved in the brain’s representation of action‐outcome associations (Chapter 3). It is likely to play a role in flexibly changing an animal’s behavior when the set of action values have changed. An experiment by Stephens et al., (1985) showed that aged rats were not impaired relative to young adults at learning that a specific place/action was rewarded in a T‐maze or operant chamber; however, aged rats were impaired at reversing these contingencies, consistent with what would be predicted of rats with dorsal mPFC lesions. This is also consistent with age‐dependent reversal impairments of item‐value associations, which rely on an intact orbitofrontal cortex (Schoenbaum et al., 2002; Brushfield et al., 2008). An experiment following‐up on these results by recording electrophysiologically from the orbitofrontal cortex during reversals observed that neurons in aged impaired rats did not change cue‐preference following the reversal, in contrast to roughly 30% of neurons that did reverse in young and aged‐unimpaired rats. Additionally, neurons in the aged‐impaired rats did not maintain sustained activity during a delay period between odor sampling and reward delivery. Neuron and network‐level physiological changes in old age The changes that take place in neurons and neuron‐networks of the mPFC are largely unknown. Following from anatomical changes observed in dorsolateral regions of the primate prefrontal cortex, examinations into neuronal and synaptic activity 152 changes have focused on these regions. In dorsolateral prefrontal cortex (area 46), an increase in the firing rates of single pyramidal cells during a depolarizing stimulation has been observed in in vitro preparations, and this effect is apparently restricted to neurons in layer III (Chang et al., 2005; also reviewed by Luebke et al., 2010). Luebke et al. (2004) also have found that the frequency of spontaneous, excitatory postsynaptic currents in layer III neurons is reduced with old age, while the frequency of inhibitory post‐synaptic currents are increased (with no apparent change in amplitude of EPSPs or IPSPs). The relationship between increased baseline inhibition‐to‐excitation and the increase in depolarization‐induced action potential frequency is not known. The network‐level significance of these synaptic data, and whether the patterns extend to medial prefrontal cortical regions, also remains unknown. Chapter Summary Although aging takes its toll on most biological tissue, the prefrontal cortex appears to be particularly sensitive to old age. Behavioral and imaging analyses have specifically implicated changes in the the dorsolateral prefrontal cortex as responsible for cognitive declines in old age; however, a number of changes have also been identified in medial prefrontal cortex (mPFC). In humans, gray matter changes can be found in dorsal regions of anterior cingulate more than in ventral regions; in rodents, this pattern appears to be reversed, with ventral mPFC regions appearing more sensitive 153 to aging than dorsal. Cognitive declines in old age related to prefrontal dysfunction, as well as the physiological declines taking place in the prefrontal cortex, may also be attributed to dopaminergic and cholinergic changes taking place with aging— modulators that play a direct and important role in the state of neural computation (reviewed in Chapter 4). High‐performing aged adults may in fact exhibit increased prefrontal engagement while performing tasks, suggesting more areas are recruited to compensate for functional declines taking place. In vitro examinations of primate dorsolateral prefrontal cortex have identified increased firing rates in layer III excitatory neurons during experimental depolarization, but decreased baseline excitatory synaptic activity. A connection between these neuron‐level changes and circuit‐level changes has not yet been made. 154 CHAPTER 6: SUMMARY OF REVIEW, HYPOTHESES, EXPERIMENT DESIGN Chapter aims and challenges A full and detailed description of the methods used for the experimental work of this dissertation has been reserved for Appendix A, and some general methods can be found in the results sections of Chapters 7‐10 where appropriate. The present chapter seeks to bridge the introductory review chapters with the experimental chapters by briefly summarizing the review, articulating a brief set of specific hypotheses, and explaining the rationale of the methods used to address the hypotheses. This chapter also contains a thorough description of the behavioral task used, including figures. One caveat to the present chapter: the presentation of a linear trajectory is in some ways a façade veiling what was more truthfully a non‐linear process. A year of iterating pilot experiments, not discussed, helped form the original hypotheses and behavioral task. Naturally, but also somehow surprisingly, after the present studies were initiated the neuroscience field continued to evolve—and as the field developed, so did the study’s emphases. Summary of medial prefrontal cortex: structure, function, algorithm, and aging Aristotle held that understanding something meant understanding four classes of “causes”: the material cause is the physical material that the thing is composed of, its formal cause is the arrangement of that material, the efficient cause is the process by 155 which that arrangement is made, and the final cause is the function, or role, of the thing. Several thousands of years later, David Marr divided analysis of the nervous system into a slightly different scheme. A brain region has a computational function, which describes the region’s role to the organism or the rest of the brain (more or less Aristotle’s final cause). This function must be implemented biologically, with specific materials and their arrangment (essentially Aristotle’s material and formal causes). Marr added a level that connects these two domains: the algorithm, or the mechanisms, by which activity of the material interacts to compute and thereby to generate the functional role of a brain region. To understand the frontal cortex, the review began by examining its material: neurons, nodes with activity states that act upon one another through synapses. Neurons could be subdivided by the neurotransmitters they used; most of these fell into excitatory and inhibitory, although the complex, slow‐acting effects of neuromodulatory neurotransmitters were also covered. The macroscopic arrangement of neurons revealed that the medial prefrontal cortex (mPFC) was connected with other regions of the brain through a number of classes of loops. Among these were loops with other regions of cortex, with the thalamus, with regions of the “limbic system” including the hippocampus and amygdala, and loops that passed through the basal ganglia network. Notable evolutionary divergence in the structure of the frontal cortex, even within mammalian species, revealed that there would be limits in generalizing the function of prefrontal regions across species; although impressive similarities could be found in the 156 mPFC, or cingulate cortex. These similarities suggest that many general principles by which the mPFC works may be learned by studying its function in the rat. The review then wound its way through what is known about prefrontal cortex function. Research in humans and monkeys suggest that concepts of cognitive control (i.e., overcoming “prepotent” decision circuits based on contextual goals) and action schemas (representation of action gestalts) are useful constructs for understanding prefrontal cortical function. Narrowing‐in on the rat mPFC, evidence was reviewed that dorsal regions have a clear but complex role in selecting actions based on their expected outcomes in a particular context. More ventral regions of mPFC may also be important for selecting feelings and emotions according to the context. A reoccurring concept in the review was the idea of “expectation,” and how the mPFC may support a particular species of expectation involved in the selection of actions and internal states. The review next examined the physiological properties of the mPFC that are thought to contribute to the algorithm of its function. This section began with a discussion of how stable communication takes place in networks of excitatory and inhibitory neurons, including a history of sustained activity and the proposed role stable activity states may play in the function of frontal cortex (the putative substrate of working memory). It was argued that neural oscillations may contribute to stable communication in cortical networks. Finally, the review covered physiological patterns in the brain known to correlate with decision making, including mPFC correlates of 157 action‐value conjunctions, uncertainty‐related neural activity in the orbitofrontal cortex, and the apparent “sweeping forward” of network states in the hippocampus and ventral striatum (but not dorsal striatum) in the comparison process. The final section of the review stepped away from understanding the “causes” or levels of analysis of the mPFC, and addressed how aging affects the frontal cortex. Functional declines with aging implicate frontal cortex. Evidence suggests that the dopamine system that is fundamental to mPFC function declines with age, and loss of neurons and gray matter in the mPFC has also been observed. Unfortunately, very little is known about how these changes in material affect changes in the mPFC algorithm, i.e., the physiological mechanisms of its function, to in turn influence behavior. In fact, it may even be the case that more of frontal cortex is used in high‐functioning, aged individuals as a way of compensating for age‐related cognitive declines. From theory to specific experimental hypotheses The medial prefrontal cortex (mPFC) is important for outcome‐ and context‐ sensitive decision‐making (Chapter 3). The algorithm by which the mPFC contributes to decision behavior likely makes use of neurons selective for actions, stimuli, and places that have a learned association with specific outcomes (e.g., Shidara and Richmond, 2002; Euston and McNaughton, 2006; Cowen and McNaughton, 2007; Hok et al., 2005, Takehara‐Nishiuchi and McNaughton, 2008). The decision process may also involve 158 representations of confidence in the stimuli or actions, which has been correlated with neuron activity in the orbitofrontal cortex (Kepecs et al., 2008) and which may also involve a putatively dopamine‐mediated increase in coherence between the hippocampus and mPFC when a rat learns which rule to use in a Y‐maze task (Benchenane et al., 2010). It is still unclear how these patterns might relate to the “sweeping forward” of network coding observed in other brain regions during decision conflict (Johnson et al., 2007; van der Meer et al., 2009, 2010). With these elements, it might be possible to construct a model of mPFC‐ mediated decision‐making. Beginning with a slight modification of the cognitive control models (e.g., Botvinick et al., 2000), the model may address the scenario of what happens when an animal must select between two or more incompatible actions associated with ambiguous or equivalent value. In this case, some neural processing sequence might take place that translates actions and stimuli associated with specific values to a representation of the conflict, and which in turn stimulates the “sweeping forward” of network activity in the hippocampus and ventral striatum to integrate the expected costs and benefits. This look‐ahead processing may eventually result in evoking a dopamine signal that reinforces (or, perhaps more accurately, sustains) the networks linked to one of the actions, resulting in the inhibition of others. Although such a model could likely be made to work, within the confines of the presently‐available data, many factors remain in deciding whether to adopt the 159 computational framework as theory. One issue is the whether or not “conflict” or uncertainty signals are explicitly generated and used by the system. Kepecs et al. (2008) report that specific neurons fire in relation to decision uncertainty; other authors recording from the mPFC of primates, however, have reported an absence of neuron firing related to decision‐conflict (Nakamura et al., 2004). Another issue is how to build a mechanistic model of decision making from its most basic elements: the neurons themselves. Many mechanistic models of cognition and behavior rely on assumptions about the behavior of neurons, usually based on small‐scale networks. Examining the interactions between specific classes of excitatory and inhibitory neurons during decision behavior may reveal that some of these assumptions are erroneous, and may help articulate the importance of the neuromodulatory loops detailed in Chapter 2. Experimentally investigating a theory of the algorithm used by the mPFC begins with an outline of that theory in the form of several, specific hypotheses and predictions: 1) The role of the mPFC in decision making is mediated by increased, coherent neural communication in the region during decision times, and this increase in neural communication is even more salient during periods of decision conflict, when contextual details of the alternative actions are evaluated. Based on the known connection between neural communication and electrophysiological oscillations, this hypothesis predicts that the 160 oscillatory synchronization of neural activity, specifically the 30‐80 Hz gamma oscillation, will increase during decision‐making 2) The involvement of the mPFC in decision‐making involves the comparison of the values or consequences of alternative possible actions. If action values or outcome predictions are represented by the mPFC, then this hypothesis predicts a transient, simultaneous activation of the neurons encoding those features for all available alternatives, followed by a reduction of activity in all but the neurons encoding the action‐set that is selected. 3) Functionality of the mPFC is reduced in old age, perhaps as a result of changes to neurons intrinsic to the network, but more likely resulting from changes in dopaminergic and cholinergic systems that regulate frontal activity. Consistent with previous results, this reduced functionality should accompany a compromised ability to overcome previously‐learned cue‐ value associations or to shift toward attending to a new stimulus modality. 4) Neural communication in the aged mPFC, particularly neural communication taking place during decision‐making, will be less stable than in young adults— 161 meaning, some process or processing failure will create increased interference in network computation. This hypothesis predicts reduced consistency in the set of firing neurons across trials in old rats, or less coherence with the gamma oscillation across trials. Justification of experimental choices: species, behavioral task, and probes Experiments to test the above hypotheses had several requirements. These requirements had to be met by a suitable animal model and a task that would both functionally engage the mPFC, and be compatible with recording neurophysiological activity. Investigations of the brain are often performed on rats (Figure 6.1; see also Shettleworth, 2009). A scientific infrastructure has built‐up around the species with respect to knowledge, availability, and laboratory materials, making it convenient for continued investigation. Why the rat? How did the precedent first begin? Its roots can be traced to The University of Chicago in the mid‐1890’s, with Henry Donaldson and Adolf Meyer identified as the individuals that most contributed to the rat’s popularization (Logan, 1999, 2005). Among the rat’s special characteristics was its usefulness for developmental research: rats are born early in development and, 162 developed relatively slowly. Logan (1999) quotes a 1915 book by Donaldson as stating the following rationales: The Albinos are clean, gentle, easily kept and bred, and not expensive to maintain. They are omnivorous, thriving best on table scraps. The span of life is about three years and breeding begins at about three months. Furthermore the species is cosmopolitan. The litters are large and may be had at any season. Their young are immature at birth. The domesticated Albino crosses readily with the wild Norway. The rat, both wild and domesticated, takes exercise voluntarily and is susceptible to training. It is also highly resistant to the usual wound‐infecting organisms. For a number of lines of study therefore, the rat seems to be a peculiarly suitable animal. Many have warned against using rat data to make inferences about other brains. Donaldson himself, during his later years, appeared to be wary of psychologist’s exclusive relationship with rats (Logan, 1999); more recently, Todd Preuss (1995b, 2000) has written in opposition of using any animal as a “standard model”. Preuss is an important figure for rat prefrontal researchers, as he has convincingly argued for the absence of a homology between the rat mPFC and the primate dorsolateral prefrontal cortex (Preuss, 1995a). 163 Figure 6.1 Use of different animal species for in vivo neuroscience investigation. Articles from the Journal of Neuroscience were compiled for four sample years, each approximately one decade apart (x‐axis; articles from 2010 include only the first six months). These articles were counted and sorted according to which species were used, presented here as proportions of all articles included in the count (y‐axis). An effort was made to exclude studies that examined neurons in culture or only examined specific genes. The figure shows that rats are more frequently used than any other species; however, this trend may be turning around in recent years due to the historic use of mice for genetic examinations combined with the recent development of genetic techniques for studying neuroscience. Recent publication of articles examining humans in Journal of Neuroscience results in part from increased technology for non‐invasive experimentation, but also from additions in the topics that this particular journal has chosen to include. Although the proportion of primate research published in Journal of Neuroscience appears relatively low compared with other mammalian species, one must take into account that these investigations can take much more time to complete. Results may also differ if a wider range of neuroscience‐related journals were examined. 164 These data were collected by University of Toronto undergraduate volunteers in the lab of Dr. Kaori Takehara‐Nishiuchi. Use of the rat in the present experiments can be justified for the following reasons: 1) historical precedent, going hand‐in‐hand with increased knowledge about its nervous system and an already established infrastructure for recording, housing etc.; 2) it is a mammal, and its brain exhibits many agreed‐upon homologies with other mammals, including the mPFC; 3) the rat is small and its normal behaviors (e.g., running) are conducive to experimental apparatuses; 4) because of the rat’s limited lifespan, it is possible to affordably study a sufficient number of aged and young adults to statistically generalize age differences across the population; 5) unlike many primates, humans in particular, the needs of the rat appear to be relatively simple and relatively easy to satisfy, even when the animals are subjected to invasive methods and highly‐controlled environments. Choice of experimental paradigm can be informed by experiments that have examined the behavioral effects of mPFC lesions. One replicated observation is that rats with damage to the mPFC have difficulty making the correct choices following a change in the required action‐set or attentional‐set (Chapter 3). It is also the case, however, that the mPFC is only necessary the first few times a rat is exposed to a change in the required action set (Rich and Shapiro, 2007). If in an intact animal the mPFC is not involved in adapting to a rule change after the first few exposures, then it becomes 165 difficult to relate the contingency of this behavioral adaptation to observed physiological changes (that is, a given rat may have too few samples for a relationship to be statistically determined). In spite of this hurdle, several authors have recently succeeded in identifying physiological changes in the mPFC that accompany behavioral adaptation to rule changes (Rich and Shapiro, 2009; Durstewitz et al., 2010; Peyrache et al., 2009; Benchenane et al., 2010). In only one of these studies, by Benchenane and colleagues, did the authors provide data that suggested why neural activity changed—it appeared to relate to a dopamine‐mediated effect that accompanied changes in the rat’s confidence of receiving reward. These studies unfortunately provide little information about the physiological mechanisms by which the mPFC’s contributes to the selection of an action or action set. A second factor that must be taken into account while deciding on an appropriate task is how to control the rat’s behavior and environment to dissociate specifically which variables physiological mPFC signals are correlated with. The need for appropriate controls is particularly salient in view of previous work showing that neural activity is sensitive to sensorimotor state (Euston and McNaughton, 2006; Cowen and McNaughton, 2009; Chapters 3 and 4). One approach to controlling behavior and environment would be to completely confine or paralyze the rat during task performance. However, even when rats are placed in a narrow chamber that limits movement, and even when they are required to keep their nose in a confined hole, movement correlates of neural activity can be observed (Cowen and McNaughton, 166 2009). A second approach is to ensure that the rats sample many different combinations of movements and sensations during the time periods of interest for decision‐making, so that decision‐activity is not confounded with the rat’s sensorimotor state. This is the approach attempted in the current task. The experimental hypotheses call for a decision task with varying levels of decision‐difficulty, and in which a rat’s movements and brain activity can be monitored while the task is performed. In view of the importance of the mPFC in extra‐ dimensional shifts, a decision task was designed in which rats had to follow a specific cue of a certain modality (auditory or visual), in the presence of a cue of the other modality. Following an extra‐dimensional reversal, rats had to learn to ignore the previously‐learned cue modality and to only follow the newly‐rewarded cue modality. To control for potential movement‐confounds, a platform was used that enabled a wide range of movements , and allowed trials to initiate when rats were in multiple different starting positions or sensorimotor situations. More specifically, the task took place on a platform with three arms and a circular, central zone. A trial of the task began when a rat entered the central zone, setting‐off one or both of an auditory cue (a ringing sound) and visual cue (a blinking LED) localized to the ends of independently randomly‐selected arms. If rats correctly localized and approached the rewarded cue to the end of the platform arm, they were rewarded with a drop of liquid food reward (Figure 6.2). The same cue continued to be 167 rewarded for an eight‐day period, at which point the extra‐dimensional reversal would take place, and rats were required to switch their behavior to approach the other cue (see Figure 6.2C, and Appendix A for more detailed methods). 168 A feeders speakers rat LED B Cues presented Outcome delivered Cue zone Feeder zones To feeder Return C Switch day ½ single cue ½ single cue ½ 2 cue ½ 2 cue ½ 2 cue ½ 2‐cue, 2cue ½ single cue ½ single cue 2‐cue 2‐cue Switch day 9 sessions training Task A Task B Task A Task B Figure 6.2 The 3‐choice, 2‐cue decision task. A: Photograph of a rat on the platform to show the proportional size and environment. B: A trial begins when the rat enters the cue zone. Two cues are presented from randomly selected arms. If the rat follows the rewarded cue, a drop of Ensure is released. If the rat reaches the end of a different arm, an error sound is presented from a speaker under the table. C: Organization of session. A block of sessions began when the cue‐reward contingencies were reversed. For the 169 following two days, rats completed trials on the 2‐cue condition, followed by four days of half‐2‐cue, half‐single‐cue sessions, followed by a 2‐cue session, and, finally, the next switch day. The final major methodological decision was the question of probes to investigate neural activity. Electrodes are currently the most accepted and used tool to detect changes in the action‐potential activity of individual neurons and the oscillating synchrony across large groups of neurons, particularly when the neurons under investigation are beneath the brain surface and the animal is awake and moving. The twisting of two or more wires allows further discrimination of the waveforms between simultaneously‐recorded neurons (McNaughton et al., 1983; Gray et al., 1995). Combining many twisted‐wire “tetrodes” into a 14 cannula array, a “hyperdrive,” enables the experimental recording of an ensemble of neurons (Wilson and McNaughton, 1993). The larger the population of recorded single‐neurons, the greater the ability to judge simultaneous activation of neurons encoding decision alternatives in the mPFC, and therefore the better the chances will be that the present hypotheses can be addressed. The process by which the electrode array recordings are transformed into analysis of neuron spiking data additionally involves many choices, most notably decisions about which recorded electrical events are action potentials belonging to a specific neuron, and whether or not a sufficient number of the neuron’s action 170 potentials were recorded to merit analyses. This process is also called “spike sorting,” or, in laboratory parlance “cluster cutting” (the process involves manually “cutting‐out” clusters of n‐dimensional points representing features of the recorded electrical events). The balance between inclusion of false‐positives and omission of true negatives during cluster cutting is a sensitive one, and biases in one or the other direction can be informed by the experimental question. Cluster cutting guidelines are included in Appendix A. Generally, strong efforts were made to keep clusters as large as possible (by spending large amounts of time merging candidate sub‐clusters and then cutting‐out only a minimal number of spikes). Clusters that showed more than ~25% of spike‐ omission from dropping below recording thresholds were not used in the analyses, clusters with any sign of dropping below recording thresholds were not used if there was some sign of recording instability. These are the choices made for how data was collected that would be relevant for the four hypotheses specified above. The ways in which the data were examined varied according to the specific question being addressed. These are described in the following chapters, organized into a chapter on behavior of young and aged adult rats during the task (Chapter 7), a chapter on the electrophysiological properties of the mPFC during task performance observed in both young and aged adult rats (Chapter 8), a chapter describing the differences observed in aged compared to young adult physiology (Chapter 9), and finally a chapter on exploratory methods investigating mPFC physiological‐behavioral relationships (Chapter 10). 171 CHAPTER 7: BEHAVIORAL PATTERNS OF YOUNG AND AGED ADULT RATS DURING DECISION‐MAKING Introduction As reviewed in Chapter 5, aged rats are slower to adapt to stimulus‐reward reversals (Schoenbaum, 2003; Brushfield et al., 2008) and slower to switch attention and behavior following shifts in the rewarded stimulus dimension (Barense et al., 2002). These deficits resemble those observed in rats with orbitofrontal and mPFC lesions respectively (Schoenbaum et al., 2003; Birrell and Brown, 2000). Although the contribution of frontal regions to normal learning rates in these tasks is clear, it seems likely that the functions can be localized to a finer temporal resolution. A rat learns over the course of multiple trials, but it may be that specific behaviors within trials provide markers for the engagement of the prefrontal cortex. In aged rats, these behaviors may be absent; or on the other hand, it may be that compensatory processes in aged rats make these behaviors even more pronounced. Scalp EEG recordings from human subjects and imaging data predict that the cingulate is functionally engaged any time there is conflict in how to respond (e.g., Botvinick et al., 2001; see chapter 3); response conflict is likely to accompany hesitation 172 behaviors, which in the rat has been referred to as vicarious trial and error (VTE; Muenzinger, 1938). The cingulate has also been implicated in error‐processing, when responses are likely to be reevaluated for reinforcement learning. Animals may react in different ways following an error, although some results suggest that a tendency to slow down and improve on subsequent trials is related to mPFC activity (Kerns et al., 2004; but see Fellows and Farah, 2005). A set of experiments was conducted to evaluate the effect age has on a rat’s ability to decide between two potentially rewarded cues, and to examine whether age influences specific decision‐related behaviors during such a task. Behavior was monitored while rats performed the 3‐choice, 2‐cue decision task introduced in Chapter 6. A trial of the task began when a rat entered the central, circular zone of the 3‐armed platform, triggering the presentation of a light and/or auditory cue from independently random arms. The rat then initiated a trajectory to the chosen arm, and when reaching the end of the rewarded feeder, was presented with liquid food reward. Only the auditory or the visual cue was rewarded during a 7‐day block of sessions, after which the rewarded cue was reversed during the middle of a session (Figure 6.1). Aged rats are impaired at localizing auditory cues, but learn to follow visual cues at the same rates as young adults 173 Both young and aged adult rats learned to approach the correct cue during the week between reversals (Figure 7.1). Learning rates following a reversal to the auditory‐ rewarded cue were slower in aged rats compared with young adults; however, learning rates were not significantly different between age groups following a reversal from the auditory to the rewarded visual cue (paired t‐test of regression coefficients of performance for each session following the first or second switch, alpha = 0.05; Figures 7.1 and 7.2). Performance of aged rats on the auditory task was reduced even in the single‐cue condition, suggesting that their deficits were not caused by interference of the non‐rewarded stimulus, but by increased difficulty navigating toward the auditory cues. Although aged rats were consistently slower than young adult rats to complete trials, the median speed of trial completion did not change over learning (Figure 7.1, bottom panels). 174 Figure 7.1 Performance and trial completion time over sessions. Top panels: both aged (blue) and young adult (red) rats improved over sessions on the auditory (left) and visual tasks (right) from the time that the reward contingencies of the two cues were switched until the subsequent switch (light blue vertical lines). Rats consistently performed better on the single cue condition (dashed lines) than two‐cue condition (solid lines). Both age groups performed more poorly in the auditory as compared to visual conditions, although aged rats were also impaired at the auditory task relative to young adult rats in both single‐cue and two‐cue conditions. Trial completion times (lower panels) did not change over sessions, although aged rats were consistently slower than young adult. 175 Figure 7.2 Learning rates over trial blocks following switch to the visual task. Proportion correct was calculated for each block of 10 trials following an auditory‐to‐ visual reversal of the rewarded cue. There was no learning rate difference between the two age groups. Further evidence for the perceptual difficulties associated with following the auditory cues can be observed by looking at the frequency at which the rats entered the non‐cued zone across sessions in the auditory and the visual tasks (Figure 7.3). Aged rats were consistently more likely than young adults to enter the non‐cued zone (2‐cue auditory task: p = 0.002; 2‐cue visual task: p = 0.0003), but the tendency to enter the non‐cued zone increased across sessions of performing the auditory task in the aged adults, and decreased across sessions of performing the visual task in both age groups. 176 This can be explained by the rats learning to ignore the visual cue in the auditory task, relying instead on the more uncertain location of the auditory cue. Figure 7.3 Proportion of trials aged and young adult rats ran to the non‐cued feeder. As aged rats learned the auditory task, they increasingly ran toward the non‐cued feeder zone, presumably due to mistakes in localizing the auditory cue (blue trace in left panel). Following a switch to the visual task (right panel), the tendency to visit the non‐ cued arm decreases gradually across sessions in both aged (blue trace) and young adult rats (red trace). Although not explicitely examined, one might expect that this decrease is not only due to learning to ignore the auditory cue, but that it might also accompany changes in the ways the rat moves its body and head around to find the presented visual stimuli. Aged and young adult rats performed more poorly, and made slower decisions, in trials with decision conflict Increased decision conflict could be operationally defined as any trial‐type that systematically caused rats to perform more poorly and to exhibit longer reaction times. 177 Since the presentation of the auditory and visual cues were independently random, the two cues were presented from the same feeder zone by chance one‐third of the time (congruent condition), and from different feeder zones two‐thirds of the time (incongruent, or conflict condition). Performance on congruent trials was high both before and following experimental switches of the cue‐reward contingencies; in contrast, performance on incongruent trials was poor following a switch, but improved over sessions (Figure 7.4). Trial completion times were slower in conflict trials (Figure 7.4, bottom panels). It was possible to split the trial completion time into decision and running periods by splitting the trial at the point in time at which the rat accelerated to the chosen feeder. The slowing observed in conflict trials was found to be almost entirely restricted to the decision period, rather than the period following the rat’s acceleration (Figure 7.5). 178 Figure 7.4 Performance and trial completion times of aged and young adult rats across sessions. Averages are across rats, for sessions following the first time rats switched to the auditory task or switched to the visual task (aged, blue; young adult, red). Performance in congruent trials (dashed lines) was consistently high, while performance on conflict trials (solid lines) improved steadily following the switch day. Trial completion times were faster on congruent trials. 179 Figure 7.5 Amount of time before and after the decision point on conflict versus congruent trials. It generally took less time for rats to make a decision than it took for the rat to run to the ends of arms following a decision. Rats took longer to make decisions when cues were presented from different feeder zones, but this conflict did not substantially affect running time after the decision was made (n = 10 young, 11 aged). Aged and young adult rats performed more poorly when required to return to the same feeder zone When rats were required to return to the arm they had just left, they were less likely to return to that feeder (Figure 7.6). 16 On trials in which the rewarded feeder was 16 It may be tempting to use the phrase “inhibition of return,” which literally refers to a reduced likelihood that an animal will orient to a stimulus or position if that stimulus or position was attended to more than half a second before. However, in the present case, the mechanisms responsible for the reduced tendency to return to the same feeder are likely different from those responsible when examining inhibition of return in primate attention and saccades: the physical position of the rats’ bodies and 180 the same feeder the rat had just returned from, performance was reduced (paired t‐test across rats; young: auditory 2‐cue, p = 0.0002; auditory 1‐cue, p = 0.0014; visual 2‐cue, p = 0.0001; visual 1‐cue, p = 0.03; aged: auditory 2‐cue, p = 0.00001; auditory 1‐cue, p = 0.000009; visual 2‐cue, p = 0.00006; visual 1‐cue, p = 0.0004) and trial completion times increased (young: aud. 2‐cue, p = 0.03; aud. 1‐cue, p = 0.006; vis. 2‐cue, p = 0.0003; vis. 1‐cue; p = 0.0006). Aged rats performing the auditory task appeared to be most affected by return‐trials, as performance remained near chance throughout the sessions between switches. sensory organs place additional constrains on returning ot the feeder. Klein (2000) provides a review of inhibition of return. 181 Figure 7.6 Rats were less likely to return to the feeder just visited. Performance of aged (blue) and young adult (red) rats was lower on trials in which the rewarded cue was presented on the arm visited immediately prior (solid lines) compared with trials in which the cue was presented from a different feeder zone (dashed lines). Trial completion times were also slower on these return trials. Performance on a trial was not influenced by recent reward history Evidence for multiple‐memory systems in the brain has suggested that the prefrontal cortex and ventral basal ganglia system support faster reinforcement‐learning rates than systems relying on the dorsal basal ganglia. Evidence for the engagement of 182 frontal cortex in the present task may be revealed by an increased likelihood for the rat to improve its behavior immediately following an error (“lose‐shift”); or, alternatively, for the rat to sustain its behavior immediately following a reward (“win‐stay”). In the present task, there was no increased likelihood for the rats to slow‐down and improve performance immediately following an error (t‐tests on post‐reward performance or decision time compared with post‐error, across multiple permutations of session‐type, or session averages, in young and aged rats, alpha = 0.05). An autocorrelation method was also used to investigate whether there was a trend for rats to modify their choice behavior within 20 trials of either a correct or error trial; this yielded similar negative results (Figure 7.7). These data suggest that the contribution of prefrontal learning systems in the present task was absent or below detection. 183 Figure 7.7 Autocorrelation of performance following correct and error trials. Plots show the proportion of correct trials in the first 1‐20 following a reward (top panels), and the proportion of trials to the non‐rewarded cue following an error (bottom). A fast reinforcement‐learning system, thought to be supported by prefrontal networks, should show short‐term effects of the reward or error on performance. There was no strong evidence that short‐term learning was involved either on the high‐performing days prior to a reversal (left) or days following a reversal (right). Summary of behavioral patterns observed in young and aged adult rats 184 All together, the most salient differences between aged and young adult rats in the 3‐choice, 2‐cue decision task were 1) reduced performance of aged rats in localizing auditory cues, and 2) slower behavior. Aged rats learned at the same rates as young adults following a reversal to rewarded visual cues. This result was unexpected based on previous work showing age‐related learning rate differences. One difference between this and these prior studies is that in the present task even younger rats took a very long time, consisting of hundreds of trials, to learn the reversed cue‐reward contingencies. The large number of trials it took for rats to learn the rewarded cue puts some doubt on the question of whether the prefrontal cortex is in fact contributing to reinforcement learning in the task, since prefrontal‐dependent reinforcement learning (thought to be based largely on outcome‐expectation) is generally thought to be more rapid than learning that involves the dorsolateral striatum. The possibility that the prefrontal cortex is not contributing to the trial‐by‐trial learning is also consistent with the absence of recent reward or error history affecting performance (described in Figure 7.6). These issues are taken up at more length in Chapter 11. Even if the prefrontal cortex is not primarily responsible for learning rates in the 3‐choice, 2‐cue task, this does not mean that the medial prefrontal cortex (mPFC) is not engaged during the task. As described in Chapter 3, the mPFC is often engaged during response conflict. Not surprisingly, all rats had more difficulty when the two cues were presented from different arms compared with when they were presented from the same arm—taking longer to make decisions, and tending to make more errors. 185 Interestingly, the rats also appeared to have more difficulty returning to the feeder zone they had just left, compared with when the rewarded cue was presented from a different feeder zone. The increased difficulty, or response conflict, of these conditions makes them useful for testing hypotheses for how the physiology of the mPFC supports decision making. 186 CHAPTER 8: PATTERNS OF PREFRONTAL NEURON ACTIVITY BEYOND THE EXPERIMENTAL HYPOTHESES Introduction As discussed in Chapter 6, solving for the algorithm by which the medial prefrontal cortex (mPFC) contributes to context‐ and outcome‐ sensitive decision‐ making requires a rich understanding of how the physiological circuit operates. Basic knowledge of how cortical networks behave in rat frontal cortex provides scaffolding for understanding how they are engaged during decision‐making. As described in Chapter 4, much of this groundwork is still missing. On the path toward addressing the experimental hypotheses regarding the role of the mPFC in decision‐making, the present chapter will describe basic physiological patterns observed in the mPFC. These observations fall into two general categories: 1) why do neurons fire action potentials (spikes) at the times that they do? Do patterns of neuron spike‐timing vary between identifiable states of the system? 2) What measurable, external variables cause variations in the firing rates of mPFC neurons? The 3‐choice, 2‐cue decision task described in Chapter 6 was designed so that a number of behavioral and environmental variables could be dissociated from one another and linked to neuron activity. The 187 ability to disentangle the different behavioral and environmental correlates of neuron firing will provide the foundation for addressing how the neurons and their interactions are involved in decision‐making, the topic addressed in Chapter 10. All results reported in the present chapter were found to be true in both aged and young adult rats, and the data were therefore pooled together unless otherwise stated. More detailed examination of some neural activity patterns reported on in the present chapter did reveal quantitative differences between the two age groups, and these are reported in Chapter 9. Additionally, region differences were found only rarely and tended to be subtle (for example, with respect to neuron coding of trial phases or between trials). Where region differences were observed, they are reported explicitly, where they were not observed, all neurons were pooled together. Based on tetrode depth and histology, two‐thirds of all neurons recorded were estimated to be recorded from the dorsal anterior cingulate cortex, the remaining neurons split evenly between the precentral and prelimbic regions. Thus, when proportions of neurons with particular activity patterns are described, these proportions are best thought of as representing activity taking place in the dorsal anterior cingulate. Although some quantitative differences in neural coding may exist between the recorded regions, these were generally below the error afforded by the relative sample sizes and the imprecision of reconstructing the trajectory of each individual electrode (except, again, where otherwise stated). 188 Part 1: Spike timing Neurons can be classified by the shapes of their action potential waveform and their autocorrelations Single neurons were recorded from the medial precentral (mPC), dorsal anterior cingulate (ACC) and prelimbic (PrL) regions of the mPFC of aged and young adult rats. By relating neurons’ action potential timestamps to one‐another using cross‐ correlations, a set of inhibitory neurons were identified by their tendency to fire immediately prior to a dip in the firing rates of nearby neurons (this method has also been used by Csicvari et al., 2003 and Bartho et al., 2004). Consistent with previous reports, the shape of the action potential waveforms in the inhibitory neurons was often narrower than those of most neurons (Figure 8.1A). Roughly 6.7% of the recorded neuron population contained similarly narrow waveforms, and these were together classified as putative inhibitory neurons. An additional group of neurons with even narrower waveforms could also be distinguished by a low preceding hyperpolarization of the extracellular action potential. This waveform shape is generally thought to be associated with the recording of an axon fiber, which is partially confirmed by the finding that these units continue to fire following local injection of muscimol (Jeremy Berry and Robert Muller, personal communication). These made up roughly 2.1% of the 189 recorded neuron population and were classified as putative axons. The remaining neurons were classified as putative excitatory neurons. Neurons could be further classified by their firing properties, as observed in the autocorrelograms of the action potentials (spikes). Based on the peak latency and decay rate of the autocorrelations, at least three categories of neuron firing could be identified which were labeled as fast‐spiking, bursting, and regular spiking (Figure 8.1B). Table 8.1 describes the number of neurons that were recorded in young and aged adults across regions of the mPFC and across these classifications. 190 Figure 8.1: Classification of single neurons. A) A comparison between the width of action potential waveforms from peak to trough (y‐axis) versus the width at half‐ amplitude (x‐axis; each recorded neuron represented by the gray dots) revealed two major grouping. The average action potential waveforms of two neurons exemplifying each of the two groupings are shown in the upper and lower insets, the associated blue line indicating their position in feature space. Inhibitory neurons identified using cross‐ correlation methods primarily clustered in the narrow‐waveform grouping (black dots). All neurons in this grouping were therefore classified as putative inhibitory neurons. A smaller group of neurons with exceptionally‐narrow waveforms were classified as axon 191 fibers. B) Autocorrelation peak‐latencies are plotted against autocorrelation slopes for all neurons. At least four groupings are revealed: a distinctive cluster of fast‐spiking neurons with peak‐latencies between 14‐25 Hz (2.6% of all neurons, of which > 60% were also identified as putative inhibitory neurons, black box); a group of bursting‐type neurons (autocorrelation peaks before 25ms with steeper slopes, 15% of all neurons, green box); a group of late‐peaking (after 45ms) neurons with steeper slopes (31% of neurons, upper orange box); and a group of late‐peaking neurons with flatter slopes (35% of neurons, lower orange box). Based on post‐hoc investigation of late‐peaking neurons and previous reports (e.g., Bartho et al., 2003), the two groups with late autocorrelation peaks were combined and classified as regular spiking neurons. Roughly 9% of neurons had autocorrelations with peak‐latancies intermediate between the regular spiking and fast‐spiking clusters and are presently left unclassified. Black dots indicate neurons identified as inhibitory using cross correlations, as in A. Inset shows autocorrelations averaged across neurons in each group, with colors corresponding to box colors. Excitatory Inhibitory axons Fast spiking Bursting Regular spiking Young PrC 462 25 1 11 62 341 Young ACC 1249 108 10 41 153 970 Young PrL 422 20 19 6 167 306 Aged PrC 48 0 0 0 11 30 Aged ACC 765 63 34 21 111 571 Aged PrL 5 0 0 0 3 1 192 Table 8.1: Number of single‐neuron spike trains of each class used in the present analyses. PrC: medial precentral (<1.8mm depth); ACC: dorsal anterior cingulate (1.8‐ 2.9mm); PrL: prelimbic (2.9‐4.0mm). Colors of each group are used consistently throughout thesis figures. Histological examination corroborated the estimated tetrode placement for each depth. Neuron spike timing oscillates at gamma and theta frequencies The activity of individual neurons, and the interactions between pairs of neurons, sometimes oscillated at 40‐70Hz (gamma) and 6‐10 Hz (theta; Figure 8.2). Fast‐spiking neurons tended to spike every 15‐25ms, i.e., 40‐70Hz (gamma), and the cross‐correlations of inhibitory‐excitatory neuron pairs also oscillated at the gamma frequency. A sample of neurons classified as burst‐firing tended to fire spikes each 100‐ 150ms, i.e., 6‐10Hz (theta). Spike trains with robust theta in the autocorrelation were rare, detectible in only around 2‐3% of the recorded population; neurons in which these patterns were observed, according to the algorithm described in Appendix A, were classified as theta‐firing. The percentage of theta‐firing neurons that were also putative inhibitory neurons (by waveform shape) was approximately the same as would be expected by chance (around 5%). In one exceptional case, two theta‐firing neurons were recorded simultaneously for several consecutive days. These two neurons tended to fire about 28ms apart, equivalent to 90o of theta (28ms). Local field potential data was simultaneously collected from the electrodes recording the neuron action potentials, and from electrodes implanted into the dorsal hippocampus. Spike‐ 193 triggered averages of the hippocampal and the prefrontal local field potentials (LFPs) revealed a consistent phase relationship with the LFP theta oscillation in both regions (the phase of theta recorded in the dorsal hippocampus was offset from that of the prefrontal cortex, although this will depend on the specific electrode position in the dorsal hippocampus); the phase relationship between theta‐firing neurons and LFP theta was consistent across days. Although one of the two neurons ceased firing during the rest period, during which LFP theta was also absent, the second neuron continued to fire at theta frequency, reduced to 6.7 Hz (consistent with “type II” theta observed by Vanderwolf, 1975). 194 Figure 8.2: Gamma and theta oscillations observed in neuron firing. A) The autocorrelation averaged across all fast‐spiking neurons, magnified at the peak, reveals an oscillation with a period around 18ms (~56 Hz). Inset is the same as in Figure 8.1A, showing the mean autocorrelations of fast‐spiking (black), bursting (green), and regular spiking (orange) neurons. B) The cross correlation between simultaneously‐recorded, excitatory‐inhibitory neuron pairs, averaged after converting to z‐scores. Once again, an oscillation of 50‐60Hz is apparent in the relative firing between the two neuron groups. Also, high connectivity between excitatory and inhibitory cells can be inferred by the relatively higher probability of excitatory neuron firing immediately prior to inhibitory neuron firing (the high peak immediately left of the zero), and relatively lower probability of excitatory neuron firing immediately following inhibitory spike (dip in activity immediately following the zero) C) The average autocorrelation of 47 spike trains (1.4% of the recorded population) that exhibited theta‐frequency activity (i.e., the theta‐firing neurons). Inset is the same figure as 8.1B, but with the position of the theta‐firing neurons in feature‐space shown in green. D) The cross correlation of two 195 simultaneously‐recorded theta‐firing neurons. In this case, phase of theta firing is offset by roughly 90o. Reciprocal excitatory‐inhibitory neuron interactions are linked to 40‐70 Hz gamma in the local field potential Cross‐correlations between neurons provide some information about connectivity between simultaneously‐recorded neurons. Previous studies have inferred mono‐synaptic connectivity between two neurons based on higher‐than‐chance probability of one neuron firing within 1‐5ms of a second neuron (indicated by a peak in the cross correlation exceeding 3 standard deviations from baseline; Csecsivari et al., 2001; Bartho et al., 2003; Fujisawa et al., 2007). Although alternative explanations may explain coincident spiking between two neurons, such as the sharing of common input, following from a precedent in the literature, short‐latency cross correlation peaks were classified as putative mono‐synaptic excitatory connections. These were found to be common between putative excitatory neurons onto inhibitory neurons, particularly when the neurons were recorded from the same tetrode. Often, the same neuron pairs showing excitatory‐inhibitory cross correlation peaks also showed inhibitory‐excitatory cross correlation troughs, providing evidence that direct, negative feedback loops are created between local excitatory and inhibitory cells (roughly 49%, 250/ 510, of inhibitory‐excitatory pairs with post‐inhibitory dips in excitatory neuron activity also had post‐excitatory increases in inhibitory neuron activity; Figure 8.2B). 196 The interactions between excitatory and inhibitory neurons could also be observed by “zooming out” to the level of local field potentials (LFPs). LFPs reflect the sum of nearby electrical sources and sinks generated by ion channel activity, much of which is the result of nearby synaptic activity. While rats performed the task, the LFP revealed a particularly strong 40‐70Hz gamma oscillation (Figure 8.3). 17 Most recorded single neurons (86%) fired at a preferred phase of the local LFP gamma rhythm during some epoch of the task (between cue and outcome, between outcome and cue, or during rest; Raleigh test: 86% P < 0.05/3, Bonferroni correction). Excitatory neurons consistently fired just prior to inhibitory neurons following the phase of highest local synaptic activity. Across all neurons, action potential frequency was reduced about 25% during the most non‐preferred phase of gamma compared with the most preferred phases of gamma (this will be described in more detail in Chapter 9, e.g., Figure 9.2B). The relationship between gamma and neuron spiking, including associated figures, is investigated in more detail in Chapter 9 in the examination of age differences. 17 Due to the proximity of the recording and reference electrodes, spatially‐diffuse, low‐frequency LFP patterns may have been underrepresented in the data, accentuating the prominence of gamma. Gamma was found to reach full power when the recording and reference electrodes were at least 1.5mm away from one‐another. Low‐frequency patterns (below 10Hz) in the cortical electrodes may also have been disguised by mechanical artifact observed during the rat’s head movements. Low‐frequency LFP patterns were therefore not examined in the present analyses. 197 Figure 8.3: Local field potentials and average power spectra. Upper left: an example of 500ms of the LFP. Middle left: the power spectrogram of the example; note the prominent oscillation evokes a peak in power at around 58 Hz. Bottom left: power spectrogram of the example after applying a whitening filter (see Appendix A). Because lower frequency signals are characteristically larger in amplitude than high‐frequency signals, a whitening filter can be used to remove this non‐specific, 1/F component so that the stronger oscillations at all frequencies stand out. Top right: average spectrogram across 100 sessions from 12 rats. The 40‐70 Hz gamma component stands out strongly against the 1/F. Bottom right: averaging the power after whitening and normalizing for each session accentuates the gamma component even more strongly, which exhibits a peak at 55 Hz (a second peak at 60 Hz is influenced by the 60‐cycle alternating current in the North American electrical grid). Neuron firing is phase‐locked to hippocampal theta In addition to the electrodes implanted into the medial prefrontal cortex (mPFC) to record spike and local field potential data, an electrode was also implanted into the 198 hippocampus, strictly to record the local field potential. The hippocampal recordings were marked by strong theta in the local field potential. Previous reports have found relationships between the phase of hippocampal theta and the timing of action potentials in the mPFC (Jones et al., 2005B,C; Hyman et al., 2005). Consistent with previous reports, nearly fifty percent of all neurons were found to exhibit differential firing between hippocampal theta phases either when the rat ran from the cue zone to the feeder zone, from the feeder zone to the cue zone, or during the rest periods (Raleigh test: 86% P < 0.05/3, Bonferroni correction). There were no differences between the distribution of phase‐locking strength (measured as the Rayleigh z‐statistic) between cue to feeder, feeder to cue zone, or rest. Significantly phase‐locked neurons in one of these epochs were likely to be phase locked in another (for example, around half of the 5% of neurons with strongest theta phase‐locking in during movement to the feeder were the same neurons with theta phase‐locking during movement from feeder to cue). The relationship between the theta‐firing neurons described in Figure 8.2 and the neurons phase‐locked to hippocampal theta (theta‐locked neurons) was not an obvious one. Most theta‐firing neurons showed significant phase‐locking (those that did not, generally had very low firing rates); but most theta‐locked neurons did not exhibit theta‐firing in their autocorrelations. In fact, the autocorrelations of hippocampal theta‐ locked neurons included regular firing and burst firing excitatory neurons as well as the gamma‐firing, fast‐spiking inhibitory neurons. 199 Regular‐firing neurons fire more slowly in ventral regions of mPFC during the task and in rest A comparison of autocorrelation peak latencies across tetrode depths revealed a relationship between recorded region and inter‐spike‐intervals, with progressively longer peak‐latencies observed in more ventral regions of cortex (medial precentral = 84ms; dorsal anterior cingulate = 96ms; dorsal prelimbic = 117ms; ventral prelimbic = 125ms; Kruskal‐Wallace test: H(3,2215) = 92.4, p = 0). Importantly, these differences persisted for the autocorrelations recorded during the rest period, suggesting they reflect intrinsic properties of the neuron firing characteristics rather than task‐related events. There was no observed regional change in inter‐spike‐intervals among fast‐ spiking (16‐21ms in young rats, median peak = 19ms) or burst‐firing (3‐25ms, median = 16) neurons. Part 2: Relationship of neuron firing to measurable external variables Neuron selectivity for trial phases and trials varies between mPFC regions The term “neuron coding” signifies that some property of a neuron’s firing activity changes in relation to an experimentally measurable variable. In order to study mPFC neural coding, it is necessary to select specific variables, dimensions of the 200 environment or behavior, over which to examine a correlation. These dimensions can be chosen according to the specific experimental hypothesis. The current chapter began with the claim that the physiology of the mPFC is largely unknown, and therefore it is beneficial to examine the behavior of mPFC neurons as widely outside of the experimental hypotheses as possible. This makes the total number of possible variables to choose from uncountably high. On the other hand, the number of statistically independent environmental and behavioral variables is not uncountable: most, if not all, environmental and behavioral factors have some relationship with one‐another, particularly in the context of the repeated‐trial behavioral paradigm presently used. Neuron selectivity in the present paradigm could be examined across two dimensions: 1) firing rate changes taking place within trials of the decision task, and 2) firing rate changes taking place between trials. Different phases within trials place distinct demands on the rat’s brain: there are specific times in the trial during which rats are exposed to task cues, times in which rats are more or less likely to turn, phases associated with the rat experiencing the disappointment of an error sound or the pleasure of the sought‐after primary reinforcer. Neuron firing changes taking place between trials adds to these distinctions: only on certain trials are rats exposed to the auditory cues, only when the rat is returning from specific feeders will it be required to turn in a particular direction, and only on certain trials will the rat have made a successful choice and be able to enjoy the food reward. 201 Two methods were used to identify the degree to which neuron population activity varied across trial phases and trials; one method measured the selectivity of single‐neurons, the other measured changes in the entire neuron population. To calculate single neuron selectivity, information measures (information‐per‐spike and firing‐rate‐information) and sparsity were computed for each cell (Skaggs et al., 1993; also used in Jung et al., 1994). The distributions of information and sparsity were similar between trial phases and trials (Figure 8.4), although median firing‐rate‐information was moderately, but significantly, higher for trial phase than trial (0.26 compared with 0.24; Wilcoxon rank sum test, p = 0). Firing rate information and sparsity measures revealed differences in selectivity between recorded regions of the mPFC (ANOVA on the log of firing rate information between regions: F(3,3306) = 47.4, p = 0; Figure 8.4B&C). Neurons recorded from tetrode depths associated with medial precentral and dorsal anterior cingulate contained, on average, more trial‐phase information than neurons recorded from the dorsal prelimbic region, which may also have contained more information than neurons recorded from the ventral prelimbic region. In addition to the single‐neuron measures of information and sparsity, a population measure of selectivity was computed as the Pearson’s correlation coefficients between the vector of firing rates across all simultaneously recorded neurons (the state vector) within a condition as compared to between conditions. State 202 vector correlations provide an alternative approach to reconstruction methods for quantifying the information in a population. 18 A comparison of the state vector correlation coefficients within trial phases (across trials) versus within trials (across trial phases) revealed neuron patterns to be more consistent within phases (within trials mean = 0.66 vs. within phases mean = 0.52; n = 163 sessions; two‐sample t‐test of mean correlation coefficients: p = 4 x 10‐44). Both the single‐unit measures of information and the population state‐vector correlations suggested that within‐trial variables were important in driving the activity of neurons, and that trial phase particularly played a large role in influencing network activity. These two dimensions are presently explored in more detail. 18 This is based on personal observation, but can also be logically derived. Neural reconstruction methods involve taking the state of neuron population activity and matching it to a variable of interest (e.g., space, movement, trial phase). The more consistent the neuron population state is within a condition, and the more different the neuron population state is between conditions, the better the reconstruction will be. 203 Figure 8.4 Distributions of information content. A) Information was calculated for each neuron for selectivity of trial phase (light blue trace) and selectivity of trial (black trace). Examples of firing maps are presented for four neurons: two neurons with low trial‐ phase information (upper left, green and red dots represent position of the rat on the two‐dimensional platform when the respective neurons fired an action potential, blue background is the total region occupied by the rat) and two with high trial‐phase 204 information (lower‐right). B) Coronal section showing the regions of mPFC that neurons were recorded, lines indicate tetrode depths that were used to distinguish each region. C) Distribution of log information for both trial phase(left) and trial (right). Mean firing rate information for trial phase (red lines) was lower in the dorsal and ventral prelimbic (dPrL, vPrL) regions compared with dorsal anterior cingulate (dAC) and medial precentral (mPC) regions. These data suggest that selectivity for trial phases was reduced in more ventral regions of the mPFC, perhaps related to the increased cognitive/movement differences between the task phases thought to be supported by more dorsal regions in comparison to, for example, emotional differences between the task phases thought to be supported by more ventral regions. Task cues evoke network inhibition, proximity to reward increases network excitation Over ninety percent of the recorded neuron population significantly differentiated task phases, firing differently either between cue and reward zones (84%) or between forward and return journeys (59%; Wilcoxon rank sum, alpha < 0.05). Neurons could be found that fired maximally at any given trial phase; however, activity across the population of all neurons was not uniform across trial phases. In most of these analyses, trial phase was broken into nine, 500ms segments beginning with the return to cue zone, and ending with 500ms during the reward or error. Neuron selectivity was highest during and immediately after outcome presentation, at the reward zones (i.e., more neurons increased firing rates near the outcome, and these neurons had higher information measures than those selective for other trial phases). Reward‐zone related firing was predominantly among excitatory, regular‐firing neurons. The average activity across the population of regular firing neurons gradually increased from cue presentation to acquisition of the outcome, with peak firing in most of these 205 neurons reaching maximum as the rat sampled the reward. In contrast, inhibitory neurons were found to fire most robustly at the cue zone, and mean activity subsequently decreased to a minimum at the reward zone (Figure 8.5; effect of neuron type on trial phase firing: ANOVA, F(8,26521) = 31.11, p = 0). The average activity of burst‐ type excitatory neurons was generally consistent with that of inhibitory neurons. The contrast between average inhibitory and excitatory activity during a trial was present even in excitatory‐inhibitory pairs that were reciprocally connected, as inferred from the cross correlograms. Thus, although activity of excitatory and inhibitory neurons was correlated in the millisecond range (Part 1 of this chapter), excitatory and inhibitory neuron activity was anti‐correlated in the range of seconds. 206 Figure 8.5 Task‐phase relationship with activity in different neuron groups. Neuron activity was calculated for each trial in each of nine discrete, 500ms trial phases (Appendix A). A) Firing rates of single neurons were averaged across trials, converted to z‐scores, then averaged within three neuron groups: putative inhibitory neurons (black), burst‐firing excitatory neurons (green), and regular firing excitatory neurons (orange). Neurons collected from all rats and sessions are pooled together. Inhibitory neurons tended to fire most when cues were first presented, and dropped to lowest levels when rats sampled the reward; regular‐firing excitatory neurons followed an opposite pattern. 207 Average firing of burst‐firing neurons resembled the inhibitory neurons. B) More excitatory neurons were selective for the period around reward, while more inhibitory neurons were selective for periods around cue presentation. C) Even among regular firing neurons, cells could be found that were selective for any given trial phase. This can be demonstrated by using a clustering algorithm, which grouped neurons according to firing rate patterns across the trial phases. Traces in the figure are averages of z‐ score‐normalized firing rates for neurons within each cluster. The color of each trace represents the number of neurons associated with each cluster (red highest, blue lowest). The trace that is most red can be observed to peak when rats were at the reward zone, signifying that a large population of regular firing neurons fired only as the rat reached and then spent time at the feeder. A second red trace can be found peaking on the left side of the figure, as the rat initiated its return to the cue‐zone; this represents a relatively large population of neurons that peaked when rats were leaving the feeder zone. A much smaller population of neurons fired selectively during the period immediately prior to cue delivery, or, in another group, fired only during the period at the decision point (blue traces). The clusters reveal several important features of regular‐firing neuron activity. First, neurons generally fired selectively during a particular phase of the decision task; i.e., no major group of neurons tended to fire identically across all phases. Second, neuron firing rate changes between trial phases tended to be positive; that is, there are not major groups of neurons which fired during most trial phases, but exhibited an inhibitory dip during one phase in particular (all clusters exhibit a peak, rather than a dip in activity). Finally, for the most part, the neuron population seems to have treated the return trajectory back to the cue zone as distinct from the trajectory toward the feeder; none of the neuron clusters exhibit two peaks during the return and decision periods, despite the fact that rats often occupied the same 2‐dimensional spatial position during the return and decision phases. Local oscillation activity showed similar patterns across trial phases as local inhibitory neuron activity As noted previously, LFPs recorded from the prefrontal electrodes as the rat performed the task revealed a stong, 40‐70Hz gamma component. When gamma power was averaged across trials and sessions within rats, then averaged across rats, patterns revealed a strong increase in gamma power at the initiation of task cues, which 208 was also when inhibitory neurons were found to fire most (Figure 8.6). This suggests that, although excitatory and inhibitory neurons may be involved in producing or modulating gamma, LFP gamma may be more dependent on the activity of inhibitory neurons; presumably the fast‐spiking inhibitory neurons. Lower frequency LFPs have a wide spatial spread; because of this, it is impossible to determine whether changes in low‐frequency components of the LFP have anything to do with processing taking place in the immediate vicinity of the electrode. As mentioned previously, a number of single neurons were found that exhibited burst activity at the theta frequency, these were called “theta‐firing” neurons. Activity of these neurons could be used as a proxy for identifying the patterns of theta specifically localized to the mPFC. Activity of theta‐firing neurons across trial phases was found to resemble the activity of inhibitory neurons and of LFP gamma, suggesting that the presentation of cues increased local oscillations and inhibition (Figure 8.6). 209 Figure 8.6 Average gamma power changes over trial phase are generally consistent with average inhibitory neuron activity. Top: Average whitened spectrograms across trial phases, colors represent gamma power averaged across all rats. During task performance there was a strong 40‐70Hz band across all trial phases, which appeared to be highest during cue presentation. Bottom:Averaged, 40‐70Hz LFP gamma power z‐ scores (blue) are overlaid with average z‐scores of inhibitory neuron activity (black) and theta‐frequency neurons (green) across session recordings (error bars = s.e.m). Activity of putative inhibitory neurons, which includes activity of fast‐spiking interneurons, followed a similar pattern with LFP gamma and with the theta neurons. Network activity is strongly driven by position and movement 210 The title of the current chapter includes the words “beyond the experimental hypotheses.” As already discussed, in order to examine neural selectivity for task variables, the variables must first be selected. The experiment and behavioral task were designed with a number of specific hypotheses in mind (Chapter 6), and these largely determined the environmental and behavioral factors that varied not only within trials (described above), but also between trials, addressed presently. Therefore, a list of eleven variables which varied between trials were selected to relate to neural activity, and these were organized according to those that fell within the domain of the current hypotheses (putatively indicating decision uncertainty or conflict) and those that were not hypothesis‐based (e.g., space, time, and movement). More specifically, hypothesis independent variables included: 1) the specific platform arm chosen by the rat; 2) the platform arm the rat had just visited; 3) trials that involved right‐turns from one arm to the next compared with trials that involved left turns, 4) the activity on the first compared with the second half of the session for non‐switch sessions, and 5) the rat’s velocity. Hypothesis‐based variables included: 1) conflict versus congruent trials, a variable that affected both decision‐time and performance (Chapter 7); 2) return‐trials compared with non‐return trials, also shown to affect decision time and performance; 3) long versus short decision‐time trials, a more direct estimate of deliberation or uncertainty that is also confounded with the former two variables; 4) trials prior to and following the reversal, which should affect expectations of reward; 5) error versus reward trials, differences between which may be attributable to a number of 211 psychological processes addressed in the following section; and finally 6) error versus reward on the preceding trial, which has been shown to influence behavior and cingulate cortex activity in human and monkey studies. The list of all variables in hypothesis‐independent and hypothesis‐based sets were not statistically independent from one another (as a simple example: trials that involved right‐turns were based primarily on which zone the rat was returning from) but each variable emphasized different behavioral and experiential factors. Some variables consisted of two categories (e.g., activity during the first vs. second half of the task), some involved three variables (selectivity for which zone the rat chose), and one (velocity) was a continuous measure. “Selectivity” for a given variable could be statistically assessed using either non‐parametric methods, or by using parametric methods after normalizing by z‐scores. Both were explored and the results were generally consistent between methods. For simplicity and compatibility with later analyses, the parametric measures on the z‐score normalized neural activities are reported here. This includes significance testing with Student’s t‐tests, ANOVA’s and Pearson’s correlations. Given the number of variables examined, it may seem intuitive to use a multivariate analysis to identify the factors that determined activity changes in the population of neurons. These analyses are implemented in Chapter 10. The emphasis of the present chapter is to instead look at the degree to which hypothesis‐independent 212 variables account for activity changes in the population of medial prefrontal cortex population, including how they may account for the apparent selectivity of hypothesis‐ based decisions. From this putatively more “hypothesis‐free” look at neuron selectivity, a number of patterns emerge—these patterns can in turn be used to build a more meaningful examination and theory of decision‐making. The proportion of neurons selective for each of the eleven variables, along with the phase of the task in which selectivity was observed, is depicted in Figure 8.7. Higher selectivity was observed for variables outside of the present hypotheses (space and movement) than for the hypothesis‐based variables. Spatial selectivity for the particular reward zone was observed as the rat approached the feeder zones in 36% of neurons (one‐way ANOVA on z‐score‐normalized data; alpha = 0.01). Also, up to 37% of neurons appeared to represent the zone the rat had just left; although this selectivity was highest when the rat reached the cue zone, suggesting that the activity may have been correlated with the specific movements the rat had to make on exiting one arm as compared to another. The interpretation that activity selective for the previous zone was related to cue‐zone re‐entry position is partially confirmed by examining neurons selective for “right vs. left turn” at the cue zone. This population was found to be a subset of the population of neurons selective for the prior‐feeder, and exhibited the same patterns of differential firing across trial phases. Finally, correlates between neuron activity and the rat’s velocity were observed at all trial phases, most robustly in 213 the middle of the rats’ return to the cue zone where the percentage of neurons significantly correlated with velocity reached 33%. Figure 8.7 Proportion of neurons with activity differentiating specific, between‐trial variables across different trial phases. A) Neuron selectivity was examined for five variables independent of the decision hypotheses. Most neurons fired differently between the first and second half of the session on the day prior to the reversal (purple trace). Approximately one‐third of the population differentiated between feeders when rats reached the goal zone (red trace). A partially overlapping set of neurons was selective for the feeder the rat was exiting, although this selectivity was strongest when 214 cues were presented (blue trace). Neurons selective for trials involving left‐ as compared with right‐turns appeared to be a subset of the neurons selective for the previously‐visited feeder (green trace), suggesting the strong differentiation between previously‐visited feeders at the cue‐zone was related to the different position and movement of rats when exiting different feeders. B) Neuron selectivity was examined for five variables within the decision hypotheses. As with the day prior to the reversal, on the reversal day most neurons fired differently in the first as compared to the second half of the session (purple trace). Relatively few neurons were sensitive to conflict versus congruent trials (blue trace) or to trials requiring a return to the same feeder versus a different feeder (orange trace). The set of neurons differentiating these variables appeared largely to be a subset of neurons firing on slower compared with faster trials (green trace) or firing more on error as compared with rewarded trials (red trace). C & D) The magnitude of population activity changes (mean of the absolute value of z‐score differences across all neurons) are shown across the same variables as in A and B. Changes in firing rates across the population, between trial types, generally followed the same patterns as indicated by the proportion of neurons that significantly differentiated trial types. One exception to this similarity was that, although many neurons fired differently between first and second session halfs (purple traces in parts A & B), the magnitude of this difference was relatively low (purple traces in C & D). In contrast to the large differences observed in space and movement‐selective neurons, less than ten percent of the recorded neuron population differentiated between conflict and congruent trials (two‐sample t‐test on z‐score‐normalized data; alpha = 0.01). Most of these neurons were a subset of the neurons that differentiated between error and reward trials (overlap between the two populations was far higher than chance; chi‐square = 0.046, d.f. = 1, p = 0; the remaining significantly discriminating neurons fell into the range of those expected by chance). Roughly 21% of neurons significantly differentiated between error and reward between cue presentation and outcome acquisition, most of these fired differentially only in the 500ms immediately prior to the rat reaching the feeder zone. However, rats also tended to run more slowly 215 during this period on error trials error (young: 28.5 cm/s vs. 26 cm/s, paired t‐test, p = 0; aged: 25cm/s vs. 23 cm/s, p = 4 x10‐15), and the degree to which the neurons significantly differentiating error and reward during the pre‐outcome period was highly correlated with the neurons firing relationship with velocity (Pearson’s correlation coefficient between z‐score differences on error and reward trials and correlation coefficient with velocity: p = 0, r = ‐0.5, a negative correlation coefficient indicates that the neurons which fired higher during error trials were also neurons which decreased activity at higher velocities). A significant correlation was also found between error versus reward selectivity and velocity across all neurons (p = 0, r = ‐0.33). In fact, many of the same neurons showed similar velocity correlates at a separate trial phase, following the outcome when rats were returning back to the cue zone (55% of the population; correlation between error and reward discrimination during the pre‐ outcome period of the significant neurons and velocity during the return: r = 0.28, p = 2 x 10‐9), despite the fact that rats tended to speed‐up following an error as compared to following a reward (young: 21.4 cm/s vs. 20.0 cm/s, p = 2 x 10‐6 aged: 17.3 cm/s vs. 14.9 cm/s, p = 10‐10). This evidence suggested that the mPFC neurons that were predictive of error might be better described as neurons sensitive to running speed. Running speed differences following error versus reward may also have contributed to the moderate population of neurons discriminating whether the rats were just rewarded or not (18% immediately after outcome, tapering to 6‐7% over the remainder of the trial). In Chapter 10, the importance of error‐trial activity as distinct from movement factors is 216 taken‐up in more detail, revealing that in fact a number of factors are likely to relate to neuron firing rate increases during error trials—including the expectations of the outcome or required movement of the outcome. In addition to conflict vs. congruent and error vs. reward firing, it was also hypothesized that neurons would be sensitive to the amount of time it took for rats to make a decision. Around 14% of neurons fired differentially between slower and faster trials at the time that cues were delivered. However, two‐thirds of these neurons were also neurons that differentiated which platform arm the rat was coming from, indicating they may be better described as position or movement‐related neurons. From these data alone, sensorimotor contributions to the differential firing in slower versus faster trials could not be ruled out. A possible explanation for neurons differentiating between different feeder zones was that the rats expected more food from one feeder as compared to another (although this was carefully controlled for, as described in Appendix A). Imbalances in reward expectation between feeders could be measured by the effect they had on the rats’ behavior: rat preferred any feeder associated with higher reward magnitude, and was therefore more likely to visit that feeder. In fact, one‐third of neurons were recorded in sessions in which the rat exhibited a significant degree of feeder preference (chi‐square of expected proportion of errors p/feeder compared with observed number of errors p/feeder, d.f. = 2, alpha = 0.05). However, of all the neurons which significantly differentiated between reward zones during the pre‐outcome phase, only 217 one‐third fired highest to the rat’s most preferred feeder and one‐third to the rat’s least preferred feeder, exactly as would be expected by chance. Also, only one‐third of significant feeder‐preferring neurons were recorded during sessions in which the rat showed a feeder preference, as was also expected by chance. Therefore, the spatial selectivity between feeder zones during the pre‐outcome period appeared to be independent of subtle variations in feeder preference. In summary, neuron selectivity was examined for both hypothesis‐based and hypothesis‐independent variables. The former category included decision and conflict related comparisons, such as whether the cues were presented from the same or different feeder zones; the hypothesis‐independent variables included a number of spatial and movement variables. Since the goal of the present chapter is to view the data beyond the limitations of the specific experimental hypotheses, an attempt was made to explain the coding patterns in the context of the hypothesis‐independent variables. This approach was surprisingly successful: neurons exhibited a high degree of zone and movement selectivity; on the surface, these factors could be used to explain the variables that were examined in the context of decision conflict. A follow‐up on this analysis is made in Chapter 10, where multivariate methods are used to reveal that the space and movement firing‐rate changes, while important, account for only a partial explanation of the neural activity in the mPFC. Moreover, the space and movement factors are themselves revealed to be important for selecting an action. 218 Position and place‐related firing are not affected by trial number, session difficulty, or the cues presented A relatively high percentage of neurons were found to be sensitive to either body position or movement during cue presentation (based on differential firing rates between depending on the feeder the rat had just returned from) or the specific feeder location the rat was approaching (based on differential firing rates between feeders in the pre‐outcome period; Figures 8.7 and 8.8). If, as previous studies have suggested, neural activity in the mPFC is influenced by associations with value, then one might expect one or both of these population firing patterns to be changed when values changed, either due to satiation or changes in the task difficulty (as during a reversal). It was not the case that feeder selectivity generally declined over a session as the rat became satiated—the difference between z‐score normalized firing rate activity in the high versus low‐firing rate feeders was no different in the first compared with the second half of the session (median z‐score differences appeared higher in the second half‐session, but this was not significant either across the whole population of neurons or across the neurons that differentiated feeder zones; Wilcoxon rank‐sum test, p = 0.25, 0.37 respectively). There was some transition that took place in the set of neurons coding the different feeders over the course of the session. Neither the degree of feeder‐specific coding, nor the degree of transition that took place, appeared to depend on whether the session was a reversal, high‐performing pre‐reversal, or low‐performing post‐reversal session (t‐test on second‐half, z‐score differences subtracted from first‐ 219 half; p > 0.2). There was also no effect of which cues were presented to the rat on the proportion of significant neurons. Figure 8.8 Neurons selective for feeder prior to outcome and position in the cue‐zone. A & B: One example neuron that fired prior to the outcome only on feeder 2. A) Platform firing rate map. The blue background represents tracker data of the rat’s location over the entire session, red dots indicate positions of the rat where the neuron fired an action potential. B) Neuron firing rates in the 500ms period prior to outcome delivery across each trial that the rat entered each feeder zone (y‐axis: trials, x‐axis: feeder zones). Note that in this particular session, the rat transitioned mid‐way between the single‐cue auditory task to the two‐cue auditory task. Although the firing rate appears to increase around the time of the task‐change, the firing rates of other 220 recorded neurons decreased over the session, so that the total degree of coding in the population did not appear to change. C & D) The firing rate map of space (C) and trials (D) of another example neuron, plotted as above. This neuron was recorded from the same rat during a session in which it performed the single‐cued visual task that then transitioned to the two‐cued visual task. The neuron fired in the cue zone and was selective for the previously‐visited feeder (which determined the rat’s position and movements in the cue zone). E) Distributions, across all neurons, of z‐score differences between the neurons’ preferred and non‐preferred feeder during the first half sessions (purple) and second half‐sessions (gray‐blue). Although neurons changed firing rate and coding properties between the session halfs, there was generally no net change in feeder‐selectivity within sessions across the population. Network activity is apparently not affected by reversals, but does change over a session It was hypothesized that some subset of neurons would fire differentially prior to the reversal as compared to following the reversal on the day that the reward contingencies were switched. Although around 75% of recorded neurons did appear to change activity between these sets of trials, this proportion was no greater than the proportion of neurons that fired differentially to the first and second half‐sessions during the day prior to the switch. The differential firing did not appear to be biased for any particular trial phase, and the total magnitude of the effect accounted for an average across neuron firing z‐scores of less than 0.2 standard deviations (in contrast to feeder location, which accounted for more than twice that; Figure 8.7, bottom panels). The non‐specific neural activity changes taking place over the course of a session were examined in more detail. Behaviorally, rats tended to gradually decrease their speed over a session. In high‐performing sessions (prior to the reversal), rats tended to 221 take several trials to reach asymptotic performance, but performance did not substantially improve after the initial ten trials. Correlations were calculated between neuron firing rates between cue and outcome and trial number during the high‐ performing session the day preceding a reversal. The firing rates of roughly 35% of neurons recorded during these sessions were significantly correlated with trial number, with more than half of these neurons (76 out of 183) showing negative correlations (i.e., decreasing their firing rates during the session). No relationship between the direction of the correlation and average firing rate or neuron type could be identified. Figure 8.9 describes changes in the rat’s velocity, performance, and neuron population activity, over each session. Figure 8.9 Changes in behavior and neuron population activity over a session. A) Sessions were divided into ten parts, each with equal trials (e.g., if the rat ran 200 trials, each decile was the average of 20 trials). The time between when young adult rats were presented with cues and when the rats entered the feeder zone are plotted across deciles (trial completion time, top red trace). The performance of young adult rats for 222 each decile is also plotted (proportion correct, bottome purple trace). Young rats generally ran increaslingly more slowly over the session, while performance was generally lower in the first decile of trials but plateaued afterward. B) Changes in neuron population activity over a session were computed by first dividing each session into deciles (ten parts with equal trials), and then comparing the population state vectors for each trial with the state vectors of trials in the subsequent decile (using a Pearson’s correlation). This revealed the degree to which the neuron population was similar from one set of trials to the next, thus providing information about when, during the session, the neuron population underwent the most transitioning. When only the pre‐outcome period was examined (blue trace), correlation coefficients revealed a progressive increase in the similarity between the neuron population activity in one session decile to the next, which would be expected if neuron activity were tracking the rat’s performance and speed (and, perhaps, expectation of outcome, which would be expected to stabilize over a session). In contrast, when the return trial phase was examined, the neuron population activity became increasingly dissimilar between subsequent session deciles, which can not be satisfyingly explained by behavior trends (black trace). Finally, correlation coefficients describing neuron population activity when the rat was in the cue zone, when stimuli were first presented, were generally lower and more consistent across the session (red trace). The suggestion of a dip in the middle bins of the cue‐onset trace hint at a change in the neuron population activity following a switch in the cues that are presented, or the cues that are rewarded, a condition that would normally take place during these trials. Chapter summary and prelude to subsequent chapters In this chapter, patterns of neural activity were examined at the scale of action potential timing, as well as at high temporal resolution, with respect to firing rate changes. The goal of this investigation, as described by the title of this chapter, was to step away from the experimental hypotheses as much as was possible. Neurons were classified as putative excitatory and inhibitory neurons based on their apparent interactions (using cross‐correlations) and their waveform shapes. Apparent connections between excitatory and inhibitory neurons (inferred by the cross correlations) suggested highly reciprocal interactions, and these seemed to go hand‐in‐ 223 hand with a prominent 40‐70Hz “gamma” oscillation observed in the local field potentials. By looking at neuron firing patterns, neurons could be additionally classified as regular‐firing, burst‐firing, and fast‐spiking; and a small subset of the regular‐firing neurons were also found to fire bursts consistently each 100‐160ms, or about 6‐10 Hz, an oscillation known as “theta.” Different neuron types had tendencies to fire at different phases of trials, presumably controlled by or controlling the different behaviors and cognitive activity taking place between different trial phases. Many neurons were also selective for variables that varied between trials, such as which zone the rat had chosen, which zone the rat had come from, how fast the rat was moving, and whether or not a trial was correct or incorrect. The examination of these different variables was split into two categories: hypothesis‐independent (primarily space and movement) variables and hypothesis‐based (generally relating to decision‐conflict) variables. On the surface, the differential firing across many of the decision‐related variables seemed to be explainable by differential firing related to movements, body position, and spatial location. This issue of how these “hypothesis‐independent” variables may contribute to decision‐making, and whether neuron activity changes extend beyond the sensorimotor and spatial explanations is taken‐up more thoroughly in Chapter 10. Other patterns stood out in the data, and the explanations for these patterns remain incomplete. This includes the observation that many neurons seem to significantly change firing rates between the first and second halves of a session, even 224 when no switch in the presented cues or the rewarded cue takes place. Behavior changes over a session; for example, performance improves following the initial trials of a session, and rats tend to slow down over a session as they tire and become less hungry. Some of the changes in neuron population activity may reflect these behavioral changes, such as neuron activity as rats approach the feeder zone. Other changes in neuron population activity are more difficult to decode, such as the progressive reduction, over a session, in the similarity of neuron population activity from one time in which the rat returns to the cue zone to the next time the rat returns to the cue zone. A more thorough interpretation of some of the current findings is reserved for Chapter 11. Another topic which was not examined in the present chapter were the neurophysiological differences between the aged and young adult rats. These are taken up presently, in Chapter 9. 225 CHAPTER 9: NEUROPHYSIOLOGICAL CORRELATES OF BEHAVIORAL DIFFERENCES (AND SIMILARITIES) BETWEEN AGED AND YOUNG ADULT RATS Introduction What happens to the medial prefrontal cortex (mPFC) in old age? How do these changes correlate with observed, age‐related behavioral changes? In Chapter 7, a close examination was made of the behavior of aged and young adult rats during the 3‐ choice, 2‐cue decision‐making task. Impressively, from one reversal of cue‐reward contingencies to the next, the aged rats were found to be in many respects on equal footing with their younger partners. Aged rats failed to keep‐up with the younger animals in two major ways: first, they had substantial difficulties in the perceptual problem of localizing auditory cues; second, they were, generally speaking, slower— slower to make decisions, slower to run, slower to react. Is there any reason to expect that these behavioral differences should be reflected in the mPFC? Chapter 4 described a number of ways in which harder decisions may selectively engage the mPFC. The topic of how the rats’ difficulty with auditory cues is correlated with mPFC physiology is dealt with only in the context of how the mPFC contributed to decision‐making and might have predicted error or rewarding outcomes, and this has been reserved for Chapter 10. The present chapter will revisit the questions of the previous chapter in the context of 226 aging: 1) Why do neurons fire action potentials when they do? 2) How do external factors related to the task cause variations in their firing rates? In examining the first of these questions, a connection between neuron‐interaction changes and behavioral slowing will be proposed. In examining the second, it will be argued that the similar performance of aged rats with young is well correlated with the high degree of similarity in neural coding. Part I: Aging affects spike timing and local circuit interactions Aged rats exhibit slower gamma oscillations than do young adults Examination of the gamma oscillation revealed that in aged rats, as in the young adults, the gamma oscillation was strong throughout the decision‐making task, reaching its peak power at the time that cues were presented and decreased when rats were at the feeder zones. Aged animals could be physiologically distinguished from young adults, however, with respect to the frequency of gamma, which was moderately but significantly slower (Figure 9.1). Young animals exhibiting a mean peak frequency of 56.4Hz (or 54.7Hz in unwhitened power spectra >50 Hz; 21) and aged rats a mean peak frequency of 53.5Hz (or 52.5 in unwhitened power spectra >50Hz; two sample t‐test, p = 0.005 whitened, p = 0.02 unwhitened). The age difference was not caused by an outlier, as tests remained significant following the removal of any individual rat. It also could not be explained by variations in electrode position, since peak gamma frequency 227 did not vary as a function of position and histology revealed a high overlap in recording locations between age groups (Figure 9.3 and Appendix B). The slowing appeared specific to the gamma oscillation; examination of the frequency of high‐voltage sleep spindles during the subsequent rest period yielded no age difference (two‐sample t‐test of mean spindle period length, p = 0.5). 228 Figure 9.1 Whitened power spectra across trial phases between age groups. Spectra (y‐axis) are averaged across trials and sessions, normalized by peaks, then averaged across rats for seven separate 500ms trial phase segments (x‐axis, each column a separate 500ms trial phase). The seven trial phases (seven columns) begin with the initiation of the return from the feeder to the cue zone, and end with the 500ms period following delivery of the outcome (these are described in more detail in Appendix A under “binned single neuron activity”). Vertical dashed lines indicate points in time when cues delivery was initiated (between “return” and “to feeder” trial phases) and the point in time when the outcome delivery was initiated (following the “to feeder” trial phase). The gap at the 59‐61Hz band represents its removal from analyses. Right edge graph displays power spectra averaged across trial phases (aged, blue; young, red). Top edge graph displays normalized mean gamma power for each trial phase. Frequency of peak gamma was significantly lower in aged rats across all trial phases, while power changes across trial phases did not differ between age groups. Slower gamma oscillations are related to changes in excitatory‐inhibitory neuron coupling and reduced firing rates in fast‐spiking neurons The majority (86%) of recorded neurons were found to fire preferentially at a specific phase of 40‐70Hz gamma during either the outbound journey, the return journey, or during rest (Raleigh test: P < 0.05/3, Bonferroni correction; outcome periods could not be examined due to licking artifact). There was no age difference in the proportion of neurons significantly locked to gamma; there was, however, a difference in the phase of gamma at which aged versus young adult neurons fired most. Treating each gamma peak (i.e., greatest local current sinks, following the electrophysiological “negative‐up” convention) as 0o (or 360o), the mean preferred phase of putative excitatory neurons in young rats was 26o ± 2 (i.e., just after the peak; circular mean and 229 95% confidence interval), and in aged rats was 18o ± 3; meanwhile, putative inhibitory neurons in young rats fired most at 50o ± 6, and in aged rats at 74o ± 13 (Figure 9.2A & B). The phase difference between excitatory and inhibitory neurons was therefore larger in aged as compared to young rats (n = 5 young and 5 aged rats in which putative inhibitory neurons were recorded; P = 0.006; two‐sample t‐test or two‐sample Watson‐ Williams test for circular data). The spike‐triggered averages of the LFPs (STAs) also revealed both an age‐ dependent reduction of gamma frequency and an increased offset between excitatory and inhibitory neuron firing. When STAs were generated from a 40‐70Hz bandpass‐ filtered signal, the mean period in a 200ms window was 18.1ms (55.4Hz) in young adult rats and 20.0ms (49.9Hz) in aged rats (n = 5 young, 5 aged, P = 0.03, two‐sample t‐test; Figure 9.2C). The magnitude of phase‐offset between inhibitory and excitatory neurons could be examined by shifting the putative excitatory neuron STAs backward so that the peaks in inhibitory and excitatory STAs become aligned (Figure 9.2D). The first peak (following the spikes, or 0ms bin) of the 40‐70Hz, excitatory STA was aligned with the inhibitory STA when it was shifted backward by an average of 3.0ms in aged rats and 1.5ms in young adults (n = 5 young, 5 aged; P = 0.002, two‐sample t‐test). These numbers were approximately the same as those derived from the relative phase preferences in the two age groups, and accounted for the increased gamma period. The temporal proximity of excitatory and inhibitory neuron firing within the gamma cycle is consistent with the hypothesis that local excitatory neuron spiking helps drive the 230 inhibitory neurons responsible for gamma. The increased offset between excitatory and inhibitory cell spiking could result in the slower gamma oscillation in aged rats. As described in Chapter 8, cross correlations between excitatory‐inhibitory neuron pairs were modulated by gamma, with excitatory neurons often firing immediately prior to inhibitory (Figure 8.2B and 9.3A). The average peak in the cross‐ correlations was reduced in aged rats (Figure 9.3A, inset). Statistically, the number of excitatory‐inhibitory neuron pairs in which the excitatory neuron significantly fired immediately before the inhibitory neuron was reduced in aged as compared to young adult animals (15% compared with 21% of pairs where the 1ms or 2ms bins were > 2 standard deviations over baseline, chi‐square, X2 = 8.1, P = 0.005). The loss of this excitatory‐to‐inhibitory drive with aging could have been responsible for the slower firing rate of fast spiking neurons in aged rats, which exhibited autocorrelations with significantly later peaks (20.1ms, ~50Hz) than those recorded in young adult rats (18.1ms, ~55Hz; 2‐sample t‐test, n = 21 aged neurons and 51 young adult, P = 0.004; Figure 9.3B). Other neuron types, burst‐firing and regular firing neurons, did not exhibit later autocorrelation peaks in aged compared with young adult rats. 231 Figure 9.2 Relationships between LFP gamma oscillations and neuron action potentials. A) Polar plots of preferred firing phase for all putative inhibitory (black) and 232 excitatory (purple) neurons. Inhibitory neurons fired at a slightly later phase of gamma than did excitatory neurons. B) Mean firing rates across phases of gamma for all cells from aged (left) and young adult rats (right) for inhibitory (top) and excitatory (bottom) neurons. The firing rate‐by‐phase histograms were first normalized by the peak firing phase and then averaged, so that the graphs represent the average degree to which neuron spike timing was modulated by the gamma oscillation (but overall rates did not differ between age grops). C) Spike‐triggered average LFPs (bandpass filtered 20‐200Hz) of all aged (blue) and young adult (red) putative inhibitory neurons show a significant increase in gamma period length in the aged rats compared with the young adults. D) Spike‐triggered average LFPs of putative inhibitory neurons (black) and excitatory neurons (purple) in aged (left, blue‐box) and young adult (right, red‐box) rats. Offsets between excitatory and inhibitory neurons, referenced to gamma, were longer in aged rats (gray bars in the left‐ and right‐side of figure centers show offsets). Although subtle, the offset between excitatory and inhibitory neuron firing in aged rats (left) is longer (wider gray bar) than for the young adult (right). Figure 9.3 Altered excitatory‐inhibitory interactions may reduce the firing frequency of fast‐spiking interneurons. A) Average cross correlation z‐scores between simultaneously‐recorded putative excitatory and inhibitory neurons reveal gamma modulation. Inset shows averages across pairs of excitatory and inhibitory neurons, in which a lower incidence of 1‐2ms offset was observed in aged animals (error bars = s.e.m.). This lower incidence may mean that it took more glutamatergic activation of local fast‐spiking neurons to elicit a response, consequentially slowing the gamma 233 oscillation. B) Autocorrelation firing rates in identified fast spiking cells, averaged for all aged (blue) and young (red) neurons. Overlaid dashed lines show these same traces after smoothing. Autocorrelation peaks were significantly later in aged neurons than in young adults. Slower gamma in aged rats is correlated with slower behavioral speed, but not slower behavior There was a significant correlation between the median time, across all trials and sessions, a rat took to reach a chosen feeder following cue presentation, and a rat’s peak gamma frequency (r = ‐0.60, P = 0.04). Significant correlations were also found between peak gamma frequency and median decision time (r = ‐0.58, P = 0.04), as well as the median time it took the rat to run from its decision point to the feeder (r = ‐0.72, P = 0.002; Fig. 9.4A). The relationship between gamma frequency and how fast a rat moved was therefore a general one, not restricted to the decision period when the rat arguably encountered higher cognitive demands. An important question was whether slower movement in aged rats, along with corresponding changes in sensory flow, were the cause of slower gamma oscillations, rather than the other way around. This alternative explanation is inconsistent with several observations. First, there was no correlation between mean, instantaneous velocity and peak gamma frequency across trial phases. Second, no correlation was observed between peak‐frequency and trial completion time in any individual aged or young adult rat (P > 0.05/12, Bonferroni correction; P < 0.05 in one rat, as expected by 234 chance; Figure 9.4B). Third, the age difference in peak gamma was also observed during a preceding rest period, when movement was minimal (P = 0.007). And finally, when examining the relationship between instantaneous velocity and the width of the gamma period during high‐amplitude gamma events (power > 2 S.D. over the mean), gamma periods were observed to be different between age groups across the range of velocities (2‐way ANOVA, age: F(1,80) = 27.42, P = 0; inst. velocities: F(7,80) = 0.68, P = 0.7; interaction: F(7,80)= 0.18, P = 1.0; Figure 9.4C) . 235 Figure 9.4 Correlations between behavioral speed and peak gamma. A) Average decision (left) and running (right) latency measures for each rat (x‐axis) were significantly correlated with the rats’ peak gamma frequency (y‐axes). Red dots represent young rats, blue aged rats. B) Three‐dimensional histograms for aged (left) and young adult (right) rats show the distribution of peak gamma frequencies across different trial‐completion‐times (between cues and outcomes). Hotter colors represent higher incidence of trials in which gamma was strongest at a particular frequency 236 (rows), and in which the trial was completed in a specific amount of time (columns, binned at 0.1s). Within rats there were no significant correlations between a trial’s peak gamma frequency and completion time. C) Examination of gamma period length and velocity during brief segments of high-amplitude gamma revealed a persistent age difference across movement speeds. Evidence for changes in local theta‐dynamics in the aged rat mPFC In Chapter 8, a subset of burst‐firing neurons in the mPFC exhibited activity at 6‐ 10 Hz; these were called theta‐firing neurons. Firing rate increases among this population of neurons was hypothesized to represent an activity state that is normally identified by the appearance of theta waves in the local field potential, but with the advantaged that it could be localized to the specific recorded electrode. Examination of theta‐firing neurons within each age group revealed a surprising absence of strong theta autocorrelations in the aged rats. An index could be computed for all neurons indicating the degree of theta modulation (see Appendix A); a two‐sampled t‐test comparing these indices between the two age groups revealed a significant degcreas in aged rats (Figure 9.5; all neurons: aged = 915 neurons, young adult = 2433, p = 0.001; neurons with average firing rates above 0.5 Hz: aged = 634, young = 1667, p = 0.04). There was no age difference when the theta indices of all neurons were first averaged within rats and then compared, possibly because the differences depended on a small subset of neurons that were not sampled in many of the individual animals. Future examination 237 will be required to confirm that this difference is maintained across rats, and whether it has a relationship with behavior or cognition. In addition to the examination of theta‐firing neurons in the mPFC, neurons were examined for phase‐locking to theta in the local field potentials recorded from the hippocampus. Despite the possible effects of age on mPFC theta activity (as indicated by the changes in theta‐firing neurons), there was no evidence that aged rat neurons exhibited any less phase‐locking to hippocampal theta than did young adults (t‐test on the log‐normalized Rayleigh z‐statistic; n = 6 aged, 6 young adult: p = 0.6 and 0.7 for to‐ feeder and return journeys respectively; p = 0.4 for rest theta‐locking). When pooling across neurons, there was some evidence that aged rat neurons were, in fact, even more phase‐locked to hippocampal theta during the rest periods (n = 915 aged neurons, 2433 young adult; p = 0.0003). It is possible that this is related to an increase in the incidence of REM sleep in the aged rats during the rest period (observed during recording sessions), during which hippocampal theta is prominent. Further analyses are warranted for examining neuron‐theta relationships in rest and waking. 238 Figure 9.5 The degree of theta‐frequency activity in neurons was reduced in aged rats. Top: distributions of an index of the degree of theta‐frequency activity across neurons (aged = blue, young = red). The distributions centered at 3.4. In neurons with scores greater than 4.3 (mean plus one standard deviation), theta oscillation in the autocorrelation was normally apparent by visual inspection. Bottom: average autocorrelations for neurons with theta‐firing scores greater than 4.3 in both aged groups. Part 2: The preservation of neural coding with age 239 Selectivity for trial phase and between‐trial variables does not differ between age groups The distribution of trial phase information in the firing rates of neurons from aged and young adult rats revealed what appeared to be an increase in the degree to which aged neurons were selective for trial phases in the decision task. This apparent effect was present when analyses were restricted to neurons recorded either in the medial precentral region or in the dorsal anterior cingulate (t‐test on log of firing rate information scores, alpha = 0.05). When information measures were averaged within rats, however, the apparent age difference was revealed to be restricted to neurons recorded in only a few of the aged animals (two‐sample t‐test between groups, p = 0.2). Although the pooled neuron data was suggestive of age effects on neuron selectivity, the possibility that there was something unique about the small sample of aged rats contributing to this effect, such as the exact position of the electrodes (previously shown to affect phase selectivity), can not be ruled out (histology from one of the rats did identify the drive as being particularly anterior). When trial‐phase related firing was examined between neuron types, aged rats showed similar patterns as young adult rats: inhibitory and burst‐firing neurons fired most immediately following presentation of the task cues, and average regular‐firing neuron activity reached its peak at the outcome. 240 Neurons whose firing rates differentiated between reward zones, trial speed, error versus reward, and session half were found in similar proportion in aged and young adult animals across the same trial phases (an example is presented in Figure 9.6). The variance of each neuron type across trials was found to be comparable between the two age groups (the coefficient of variation exhibited skewed normal distributions with apparently equal medians between age groups, significantly differing between neuron type; two‐sample t‐tests across cells or between rats consistently revealed higher than 0.5 probability that the distribution means were equivalent). Neuron population correlation measures also failed to find differences between age groups in the consistency of the neural code between trials (Pearson’s correlation between the state of active neurons in each 500ms trial phase bin—both aged and young adult rats showing the same differences in correlations between trial phases). 241 Figure 9.6 Comparison of aged and young adult neuron selectivity for chosen feeder zone. Left: the proportions of neurons found to significantly differentiate between the feeder zone that rats chose and entered were similar in aged and young adult rats. A slight increase in the number of neurons coding for the feeder zone was observed in younger rats during the outcome; however, the magnitude to which the neuron population differentiated the zone during this period of time, or during any other trial phase, was not significant (right figure) . Chapter summary In Chatper 7, behavioral analyses revealed many similarities in how aged and young adult rats behave in the 3‐choice, 2‐cue decision task. This similarity in behavior was consistent with a similarity in neural coding. There was no clear difference in the proportion of neurons differentiating specific task variables between aged and young adult rats, or in the degree to which the network as a whole differentiated between specific task variables. One behavioral change that did accompany aging was a slowing‐ down of behavior, including decision‐making. A salient physiological change also had to do with a slowing down: the slowing‐down of neural interactions, as observed by the reduced frequency of the gamma oscillation. The implications of this neural slowing are discussed in Chapter 11. 242 CHAPTER 10: INVESTIGATING THE PHYSIOLOGICAL BASIS OF THE MEDIAL FRONTAL CONTRIBUTION TO DECISION‐MAKING Introduction Physiological correlates of decision‐making can be thought of in terms of coding future, present, and past. Decisions can be made based on prediction of the future, as an animal speculates on possible expected outcomes and the costs of achieving those outcomes. In other cases decisions may be more tightly bound to the present‐tense, with the immediately available actions, stimuli, and places assigned specific values that drive behavior. If the behaviors are well‐learned, then value itself may be removed from the equation, as stimuli can become directly associated with specific movements, and movements associated with one‐another. Activity related to the past comes into play when the values or outcomes of a decision must be learned or re‐learned. The process by which something is re‐evaluated may involve activation of the episodes that brought the animal to where it is, or involve continued activation of the expectations that led‐up to the outcome. In Chaper 8, a number of movement and position‐selective neurons were identified, including neurons that strongly differentiated trial phase, neurons that were 243 selective for reward zones prior to and following outcomes, and neurons firing in the cue‐zone selective for which feeder the rat had previously visited. It is possible that the computational function of these responses is to maintain a representation of the rat’s current state. This possibility is further supported by the observation that neurons can be found that are selective for each trial phase, whether it is during the decision period and entry into the feeder zone, or the presumably less interesting journey back from the reward to cue zone. In order to attach value or behavioral relevance to a place or action, it is necessary for the system to be able to represent places, positions, and actions, all of which are supported by the data collected here. The question of whether neurons represent activity related to the rat’s future state, or possible future state, is investigated in the current chapter. The idea that these representations might also influence the behavior of rats is also examined. In the first part of the chapter, it will be argued that some of the neural activity observed immediately prior to the rat reaching a feeder, activity that differentiates whether the rat is making a correct choice, can be classified as expectation and distinguished from other factors that correlate with expectation, such as movement or velocity. In the second part of this chapter, an examination will be made of prospective activity as a rat enters the cue zone. It will be argued that this activity might represent a goal that contributes to the rat’s choice. That is, the prospective activity prior to a rat entering a feeder arm may, in fact, provide a neural basis for a goal representation that directs the rat to the chosen feeder arm. Paradoxically, because of the simplicity of the task, the 244 activity putatively related to the representation of a spatial goal is associated with incorrect choices. Part I: Expectation activity during the pre‐outcome period Neurons with differential firing prior to errors, compared with prior to rewards, included neurons coding for velocity, outcome‐expectation, and feeder value To establish that neural activity is prospective, it must be demonstrated that neuron firing patterns are not dependent on the actions an animal is currently executing, or dependent on stimuli that are presently available. Many experiments achieve this by introducing a “delay” period during which no stimuli are presented and the subject is required to not move. In the current task, no such delay period was introduced, which presented a challenge for identifying prospective, or expectation‐ related, activity. In Chapter 8, neurons were identified that exhibited different firing rates prior to an error on incorrect trials, compared with reward on correct trials. It was also observed that rats tended to move more slowly when reaching the outcome on an incorrect trial as compared with a correct trial. This raised the question of whether the differential firing rates prior to error versus reward were due to neuron sensitivity to velocity changes. To evaluate whether the neurons were generally velocity sensitive, firing rate‐velocity relationships were examined in other trial phases, such as after the 245 rat left the feeder zone, prior to the subsequent cue presentation. A strong correlation was observed between the degree to which neurons fired differentially between incorrect and correct trials before the rat reached the outcome, compared with the degree to which the neuron activity was correlated with running speed following the outcome (neurons significant for pre‐outcome error vs. reward: r = ‐0.3, p = 10‐8; all neurons: r = ‐0.14, p = 10‐16; see also Chapter 8). The neuron firing relationship with velocity is, in itself, interesting. Further examination revealed that many of the velocity‐sensitive neurons increased firing rates when the rat moved more slowly (return‐phase mean velocity correlation coefficient: ‐ 0.03, t‐test, p = 2 x 10‐16; similar for pre‐outcome phase). Theories for why neurons may fire more during slower running speeds are discussed in Chapter 11, including the intriguing possibility that these neurons may have played a role in slowing the rat down. This might be predicted by cognitive control theories that hypothesize that conflict or expectation of error might cause the animal to slow down, in order to increase subsequent accuracy. A more immediate issue is the question of whether velocity is the only variable that determines the pre‐outcome, correct versus incorrect trial firing differences. The correlation between neural activity related to incorrect choice and also to slow movements raises the question of whether all incorrect‐correct trial activity differences are accounted for by velocity. This would not reflect “prospective” activity for the expected outcomes (error compared with reward). To examine the degree to 246 which neuron activity during the pre‐outcome period was predictive of the outcome itself, the observed “incorrect versus correct” activity could be compared with the neural activity during the outcome. A significant correlation was observed between the pre‐outcome, error‐versus‐ reward activity and the error‐versus‐reward activity during the outcome (for neurons significantly differentiating error and reward in the pre‐outcome period: r = 0.3, p = 3 x 10‐7; for all neurons: r = 0.14, p = 10‐15). This relationship is inconsistent with the possibility that all incorrect‐correct neural activity is related to velocity: although rats moved more slowly as they reached an error, compared with reward, they moved more quickly after they were presented with an error, as compared with reward. These results can be summarized by saying that while many neurons fired more on incorrect trials because they were velocity‐sensitive, other neurons fired more on incorrect trials because they predicted the outcome, or alternatively predicted the action that would be required when rats encountered the different outcome. In addition to velocity and outcome‐expectation, there is a third variable which may also be responsible for differential activity on incorrect compared with correct trials. In some sessions, rats exhibited a feeder preference, quantified as the degree to which they made errors by entering one specific feeder over others (it is also possible to examine feeder avoidance, as the degree to which rats made errors when one specific feeder was rewarded). The appearance of a feeder preference suggests that, to the rat, one reward zone was higher in value (or lower in cost) than another. Prevous studies, 247 reviewed in Chapter 3, suggest that neuron activity is sensitive to the value of locations or actions. During correct trials, rats are expected to visit each feeder zone in equal proportion, and therefore rats will visit the higher‐valued location (when one exists) on approximately one‐third of the trials. The expression of a feeder preference means that the rat enters one “preferred” zone significantly more frequently, more often than one‐ third of the choices on any given trial. On average, the neurons coding for high‐valued locations may be more active on the incorrect trials, since the rat is spending more time during these trials in the high‐valued locations. Close to sixty percent of neurons that fired differently between error and reward trials also fired differently between different feeder zones (alpha = 0.01 for both dimensions). In fact, neurons with increased firing rates on error trials, relative to reward trials, were found disproportionately to be selective for the feeder the rat preferred most (100 neurons, or 48%, compared with 70 expected, chi‐square = 19.3, d.f. = 1, p = 10‐5; methods for determining feeder preference are described in Chapter 8 and Appendix A). This result suggests the possibility that some of the neurons, if only a minority, increase activity on error trials because they are coding for the high‐valued feeder zone. To add this observation to the two previous observations in this section: some neurons may differentiate between incorrect and correct trials because the rat moves more slowly, some may change their activity differentially because they represent the expectation of an outcome or the action that is required at the outcome, 248 and some neurons might encode the value of the feeder, which the rat visits more frequently on error compared with reward trials. To summarize, the data presented here suggests that neural activity differentiated correct and incorrect trials because of: 1) velocity, 2) expectation of outcome (or outcome‐related actions), or 3) because they were value‐sensitive. The more tedious approach to teasing apart these factors is to examine the correlations between them. This was done, and the groups of neurons coding for one variable compared with another were found to be independent, but not distinct populations. The proportion of neurons that fired in relation to velocity that were also neurons the fired differently to error and rewarding outcomes was not different from the product of the proportion of all velocity‐correlated neurons and all outcome‐differentiating neurons; that is, the overlap between the populations was approximately the same as would be expected by chance. An alternative approach to teasing apart these factors, which avoids the combinatorics of examining each pairwise relationship, is to use analytical methods to extract the correlations. These methods generally fall under the heading of factor analyses, the most common of which is principle components analysis (PCA). This technique not only separates the maximally‐independent hidden variables in the data (that is, the sets of correlated variables), it also sorts them according to the degree to which the factor contributes to the total variance between variables. 249 Principle components analysis can be used to tease apart variables contributing to neuron firing changes between trials In the previous section, correlation methods were used to determine that at least three different factors contributed to predictive firing on error compared with rewarded trials. Part of the process by which variables were distinguished from one another was to examine the degree to which firing properties generalized across trial phases. Each trial consists of multiple phases, five in particular can be used for used for examining between‐trial comparisons related to decision‐making (“return” to the cue zone, “at cue zone”, “pre‐outcome”, and two post‐outcome phases; Figure 10.1). Principle components analysis (PCA) was used to identify how neuron firing varied among the trial phases and between‐trial variables (e.g., reward versus error trials, or fast‐moving versus slow‐moving periods). Figure 10.2 demonstrates the use of PCA in the present context, revealing results consistent with those observed in the previous section, but with the benefit of a larger‐picture view of how neuron population activity in the mPFC differentiated between the variables that were chosen. 250 Figure 10.1 Position of rats on the task platform during five trial phases chosen for analysis. A) Schematic representation of a rat’s position and direction during an example trial during each chosen trial phase. Trial phases include 1) 500ms during the middle of the return from the feeder zone to the cue zone (blue arrow), 2) 500ms at the cue zone immediately following cue presentation (purple arrow), 3) 500ms immediately prior to reaching the goal zone (red arrow, referred to as “to feeder”), 4) 500ms following initiation of presentation of the outcomes (black arrow, “outcome 1”), 5) a 500ms period following “outcome 1”, in which rats began reacting to the presented outcome, either by licking at the feeder tube when rewarded (green arrow), or initiating a movement back to the cue zone following an error (gray arrow). B) Actual position of the rat during each of the trial phases, points indicating rat positions and colors matching arrow colors in part A. The first principle component (55% of the variance) describes the most salient factor that determined neural activity between trials and trial phases: whether a rat just made an error or was just rewarded. The dramatic difference in population behavior between error and reward outcomes is preceded, prior to the outcome (“to feeder” in figure), by a more minor difference in population behavior between error and reward trials. Since this is apparent in the same principle component, it implies that the same neurons that responded differently to error or reward also responded differently, to a 251 much lesser degree, to error and reward in the pre‐outcome period. This means that the same activity state that differentiated between error and reward outcomes exhibited predictive firing prior to reaching those outcomes. The relationship between outcome activity and pre‐outcome activity in the first principle component appears faint, but can be quantified using three approaches. First, a Pearson correlation can be calculated between the pre‐outcome and outcome periods, as performed in the earlier analysis. Second, only the neurons significantly differentiating correct and incorrect trials in the pre‐outcome period can be selected, and the value of the first principle component coefficient can be shown to significantly differ from zero (mean = ‐0.2, p = 0.001), or, the most direct use of the principle components. Third, neurons with exceptionally strong first principle component coefficients can be identified, and it can be demonstrated that the pre‐outcome, incorrect vs. correct trial activity is significantly different from zero (the 15% of neurons with lowest coefficients, error vs. reward z‐ score difference = 0.1, p = 3 x 10‐16). All together, these analyses show that differential activity in the pre‐outcome period between incorrect and correct trials represents an activity state in the mPFC that predicts the activity state presumably evoked by the presentation of the outcome itself after the rat reaches the feeder zone. The second principle component in the figure reveals a mis‐match between activity as the rat reaches the feeder compared with when it reacts to the outcome. The mis‐match follows the pattern of the rat’s velocity: as the rat reach the feeder it an error trial, it slows down, but speeds‐up again soon after it hears the error sound. This 252 component also exhibits correlations with velocity at all trial phases examined, suggesting that the component is the velocity‐correlate discussed in both this chapter and in Chapter 8. The next two components exhibit activity differences between reward and error trials even as the rat is first exposed to the cues. These components will be investigated more thoroughly in the next section. 253 Figure 10.2 Principle component coefficient matrix describing neural activity related to outcome prediction versus velocity. A) The degree to which neurons differentiated error versus reward, or velocity, in each of five different trial phases (rows in lower plot, 254 with stronger hot or cold colors representing stronger differentiation of the variable) was examined by calculating the principle components of the differential activity for each variable across all neurons (each principle component represented by a separate column). Top line graph shows the relative degree of variance contributed by each component, ordered left to right as in the lower plot. The left‐most column, bordered by a dark red dotted line, represents the factor, or component, contributing to the most variance with respect to the differential activity of the variables used: the first principle component. This is expanded in part B. For example, the dark blue square in A corresponds to the large bar on the top‐right of B—both indicating that the most variance in the neuron activity is determined by neural response differences when a rat has just encountered an error sound compared with when it begins consuming food at the feeder zone. Moving up by two boxes in A, or to the left by two bars in B, differential activity can also be observed in the pre‐outcome (or “to feeder”) trial phase. This difference is in the same direction as during the outcome period, meaning that the neuron firing behavior that differentiated between reward and error during the outcome also, to a much smaller extent, differentiated between correct and incorrect trials prior to the rat reaching the outcome. Note that whether the boxes of a particular column are colored blue or red (or whether the bars in B are upward‐ or downward‐ going) is arbitrary, the important variable is the strength of the cold or hot colors (blue or red) and the relationship between colors within a column. For example, if one square is blue, and the one below is red, then the component has identified an anticorrelation between the two variables; however, if one is blue, and the adjacent square in the next column is red, no inferences can be made. As with the first principle component, or first column in A, the second column also shows differential activity during the pre‐outcome (“to feeder”) trial phase. This component is expanded in C. Unlike the first principle component, where neural activity differentiated between reward and error, but did not appear to consistently differentiate between different velocities (lower graph of B), the second principle component appears to represent sets of neurons which were consistently sensitive to velocity (lower graph of C). This component reflects neuron velocity correlates: the rat continued to slow down on error trials while entering the feeder zone, before speeding up again (switch from red to blue at time of response, blue in velocity rows reflect higher activity with lower velocities across all trial phases). In the forth column, representing the forth principle component, error or reward prediction begins even earlier, at the cue zone. The patterns associated with this component are taken up in more detail in the second half of the present chapter. Part II: Prospective activity during the trial‐initiation (cue‐zone) period 255 Differential representation of cued and non‐cued, non‐entered feeders is either absent or below detection threshold The preceding section described evidence for prospective coding: neural activity that predicted outcome (more specifically, presence or absence of reward) was observed before rats reached the feeder zone. Because the activity difference was not robust when rats were at the cue zone, it seems unlikely that the differential activity played a role in the rat’s decision for which feeder to visit. Decision‐making was more likely to take place at an earlier phase of the decision task, when rats first entered the cue zone. In Chapter 8, a number of neurons were identified that fired selectively at particular feeder zones. If the mPFC is capable of representing feeder zones in the present tense (as already demonstrated), then it may also be the case that the mPFC represents the possibility, or availability, of entering the same zone prior to entry into that zone. Such simultaneous activation of the cued feeder arms (that is, the feeders that may contain reward) may be the basis of the comparison that might take place, as hypothesized in Chapter 6. In every conflict trial of the 3‐choice, 2‐cue task (comprising two‐thirds of all trials in the 2‐cue condition), one feeder zone is cued and rewarded, one zone is cued but not rewarded, and a third zone is neither cued nor rewarded. If rats are considering entering the cued but not rewarded zone, and if this consideration is represented by neuron firing in the mPFC, then neurons selective for the cued feeder zones, regardless 256 of reward, should show increased average firing rates relative to the neurons selective for the non‐cued feeder zones. Across all zone‐selective neurons, there was no significant increase in neural activity when the neuron’s preferred zone was cued but not entered, as compared with when the neuron’s preferred zone was neither cued nor entered (paired t‐test of z‐score normalized firing rates: p = 0.34 pre‐outcome, p = 0.38 around the decision time, and p = 0.39 at cue‐onset period; Figure 10.3). A search for individual neurons with significantly different firing behavior between the non‐entered cued versus non‐cued zones yielded less than 1% significance, well within the realm of chance using an alpha of 0.01 (two sample t‐test on z‐score normalized firing rate). Principle components analysis found no correlates that distinguished neurons firing in relation to the non‐ entered cued versus the non‐cued arms. A cued – non‐cued 0.06 entered – non‐cued *p = 0.0001 B z‐score difference 0.05 non‐cued, non‐entered 0.04 0.03 0.02 0.01 cued, non‐ entered p = 0.4 0 -0.01 entered Figure 10.3 Neuron activity related to the cued but not entered arm was no higher than activity related to the non‐cued, not entered arm. A) A neuron’s zone of high 257 selectivity was determined as the arm in which the neuron fired most. It was hypothesized that zone‐selective activity would increase when that zone was cued, even if the zone was not entered. An examination revealed that neurons selective for the cued but not entered zone tended not to fire any more than neurons selective for the non‐cued zone. This could be contrasted with the increased neuron activity related to the entered zone. B) A schematic description showing that neuron activity was only prospective for the chosen zone (red arrow), but not the non‐chosen zones. Prospective activity is dealt with in more detail in the following section. Neuron firing during the cue‐zone period is higher on incorrect trials compared with reward trials The number of neurons which exhibited significantly‐different firing between correct and incorrect trials was extremely low during the 500ms interval when cues were initially presented (3‐4%, accoding to t‐tests, reported in Chapter 8). This differs from the observation of predictive, error‐versus‐reward activity that occurs immediately prior to the rat reaching the feeder zone (reported earlier). Activity patterns during the period of time when cues were initially presented, however, are more likely to influence the actual choice that is made. While significant selectivity appeared to be minimal when each neuron was examined independently, when neural activity was pooled across all neurons for incorrect versus correct trials, a significant increase in activity on incorrect trials was found compared with correct trials (p = 0.01, mean difference between z‐scores: 0.008). When pooled, the effect appears weak; however, when the activity is examined in specific classes of neurons or sessions, larger differences emerge. For example, the activity differences between incorrect and correct trials appeared to 258 be restricted to neurons in the anterior cingulate cortex (Figure 10.4 A), and specifically the burst‐firing neurons in that region (Figure 10.4 B). Figure 10.4 Burst‐firing neurons of the dorsal anterior cingulate cortex were more active during cue‐presentation on incorrect trials. Across the population of all recorded neurons, a significant increase in activity was observed during the cue‐ presentation period on error trials compared with rewarded trials. The z‐score normalized activity of each neuron was compared between error and reward trials for each of eight trial phases (trial phases are described in detail in Appendix A, in the section titled “binned single neuron activity”). A) An increase in error‐trial activity was observed particularly in the population of neurons recorded from the dorsal anterior cingulate (middle). A notable change in population activity amoung neurons recorded from the prelimbic cortex was also observed during rewarded outcomes (bottom, right side of plot). Error bars are standard error of the mean. B) An increase in error‐trial activity when cues were initially presented was primarily observed in the burst‐firing neurons of the dorsal anterior cingulate (middle), rather than the inhibitory (top) or regular‐firing (bottom) neurons. All neuron types appeared to distinguish between the different outcomes following outcome delivery, with inhibitory and burst firing neurons increasing activity on error trials, and regular‐firing neurons tending to increase activity on reward trials. 259 The principle components were computed to compare how neural activity changes related to a list of variables of interest (the variables of interest were: 1) error vs. reward trial firing, 2) velocity correlation, 3)selectivity for the chosen feeder, 4) selectivity for the feeder chosen on the previous trial, 5) selectivity for the first versus second half of the session, 6) conflict vs. congruent trial firing, 7) the level of performance during the session, and 8) the feeder preference; all of which were examined across five trial phases). This analysis revealed that the increased cue‐zone activity on error trials was related to other factors, including higher performance and sessions in which rats exhibited increased feeder‐zone preferences. Determining the variables that were associated with increased activity during error trials was an important step in determining the reasons for the activity increases. A number of hypotheses could be generated to explain the increased activity on incorrect as compared to correct trials. For example, neurons may have been predicting the outcome (or action associated with the outcome); alternatively, neuron activity changes may reflect velocity differences. The analyses described earlier in this chapter, however, revealed that outcome prediction and velocity likely played minimal roles in incorrect versus correct activity when rats were at the cue‐zone (seeFigure 10.2 B &C). Other hypotheses include the following: 260 1) On error trials, rats may have moved around more to examine their choices. Because neurons in the mPFC are sensitive to movement and body position, more neurons would be observed to fire during this period of time. This would predict that the effect is due to neurons primarily active during the cue‐zone or in other circumstances in which the rats were moving in the same ways (i.e., not during the pre‐outcome period, when the variance of movement was reduced). 2) On error trials, rats may have moved around less, spending less time and passing through the cue zone more quickly. In this case, the increased firing is related to the approach to the chosen reward zone. This would also predict that the effect is caused by neurons which also fired strongly during the pre‐outcome period. 3) Neuron activity during cue‐presentation may be a representation of the possible places a rat will enter or goals that are considered. In this case, the increased firing on error trials reflects the increased number of goals the rat considers, corresponding to more total neurons active. 4) Increased neuron activity on error trials may be a product of the rat using mPFC‐mediated, goal‐directed behavior toward a specific feeder, rather than to following its learned habit. In this case, neuron firing would be expected to increase when rats are running toward the preferred reward zone, independent of whether or not the reward zone is cued. 261 These explanations can be dissociated from one‐another by examining the influence of several critical variables. First: if individual neurons are associated with specific goals or movements, and the increased activity during error trials is related to an increase in the number of neurons active when rats contemplate multiple zones or exhibit multiple movements, then the effect should be reduced when trials are restricted to only those in which rats enter a specific feeder zone. That is, the analyses can identify the feeder in which a neuron fires most, and then exclusively examine reward versus error firing when the rat runs to that feeder. This will help to distinguish between the above odd‐numbered and the even‐numbered hypotheses. Further distinguishing between hypotheses 1 and 3 might be achieved by examining the relationship between neuron firing on error trials and actions or goal zones respectively. Distinguishing between hypotheses 2 and 4 can be achieved by examining the relationship between neuron activity and the speed at which the rat makes a decision, as well as the the relationship between neuron activity and the preferred feeder. When this has been done, the hypothesis that gains the most empirical support can be further confirmed by adding relevant variables to the analyses. Convergent evidence suggests that neuron activity during the cue‐period relates to goal‐directed action selection 262 A key variable in dissociating hypotheses for why neuron activity in the mPFC was increased during the decision period is whether or not the increased activity is caused by an increase in the number of neurons active, or whether it can be observed looking exclusively at the neurons that are selective for the rat entering a particular feeder arm. One important issue for this analysis is how to determine arm selectivity; that is, a neuron may exhibit increased firing on one feeder arm when considering only the period of time when the rat was consuming reward, but may exhibit increased firing on a different arm when only the pre‐outcome period is considered. This issue was addressed by considering the selectivity of all possible trial phases (Figure 10.5). Importantly, the increased neuron activity on error trials during cue presentation remained after restricting analyses only to the trials in which each given neuron fired most. The effect was strongest when trials were restricted according to neuron selectivity during the pre‐outcome period (p = 5 x 10‐7, z‐score difference = 0.03). Additionally, there was no significant increase of neural activity on error trials when trials were restricted to each neuron’s non‐preferred zone (Figure 10.5 B). Together, these results suggest that neuron activity during the cue‐presentation period was not only higher on error trials, but also that it was higher because the neurons were firing similarly to how they would fire just prior to the rat reaching the feeder zone, specifically for that feeder zone. Such activity could potentially be caused by the rat sprinting through the cue‐zone to reach the end of the feeder arm on the error trials; however, analyses revealed that the tests were equally significant when only trials 263 slower than median were considered. Perhaps more powerfully, the finding also remained significant when only trials in which rats took an exceptionally long time to make a decision (more than 750ms) were considered. Additionally, the effect was not due to high neuron firing in only one or a few error trials, since the result remained highly significant following the removal of the highest firing trials (p = 2 * 10‐5, z‐score difference = 0.02). A likely explanation, therefore, is that the selection of a particular feeder arm is made earlier in the mPFC on error trials, and that this decision contributed to, but was not solely responsible for, the rat’s behavior. Figure 10.5 Increased activity on incorrect trials after restricting trials according to whether the rat entered the neuron’s highest‐firing feeder arm. Z‐score differences between incorrect trials and correct trials are calculated for all neurons and averaged. Analyses only considered trials in which rats entered the zone the neuron fired most in, and this was determined separately across eight different trial phases (the “reference” phases, along the y‐axis). Each box is therefore the average activity difference between incorrect and correct trials across eight trial phases (x‐axis) when considering only the 264 zones neurons fired most in (determined according to differential firing in each of eight reference trial phases: y‐axis). Hotter colors represent high incorrect activity relative to correct, cool colors represent high correct activity relative to incorrect. The highest error‐reward activity was observed when cues are first presented at the cue zone (the first “to feeder”), when selectivity was determined based on differential firing at the pre‐outcome period (arrow). The graph on the right is the same as the left, only in this case trials were restricted according to the zone the neuron fired the least. In this case, significant error‐reward firing is eliminated, suggesting that the effect is zone‐specific. Together, these data suggest that the medial prefrontal cortex increases activity on error trials at the cue zone, and that this increased activity is feeder‐zone selective and prospective for the chosen zone. In other words: on error trials, the medial prefrontal cortex may represent a specific feeder arm, and this may direct the rat’s decision to that arm. To summarize: the observation was made that neurons are more active at the cue zone on trials in which rats make an error, compared with on trials in which the rat makes the correct decision. Several hypotheses were considered to explain the effect, each making different predictions about what other factors might correlate with the increased activity on incorrect trials. One variable that helped discriminate between hypotheses was the degree to which the increased neuron activity on error trials was caused by an increase in the number of neurons that were firing, including neurons selective for the non‐selected feeder zones, or alternatively an increase in the firing rates of neurons that were selective for the chosen feeder zone. Restricting the analysis only to trials in which the rat entered the feeder zone the neurons were selective for revealed that the latter explanation provided a better fit to the data: increased neuron firing was observed in neurons selective for the chosen feeder, but not observed in neurons that generally fired less for the chosen feeder. Additionally, the increase in 265 neuron activity was strongest when zone‐selectivity was determined by how the neurons fired during the pre‐outcome period, suggesting that perhaps the increased activity was related to the mPFC selecting a feeder zone, even if the rat itself did not immediately do so (in fact, the effect was still present even when rats took a particularly long time to make decisions). These data point to the fourth hypothesis listed above: increased neuron activity on error trials may be a product of the rat using mPFC‐ mediated, goal‐directed behavior toward a specific feeder, rather than following its learned habit. This hypothesis might further predict that the increased error‐trial activity would be magnified further in cases in which the rat exhibited a clear preference for a particular feeder, as such a preference might suggest the existence of actions that were based on goal locations, rather than cue‐following behavior. Restricting the population to the neurons selective for the rat’s preferred feeder zone (roughly one third of the total population, or 1164 neurons), amplified the degree of observed incorrect‐trial population activity increases (p = 10‐12; mean difference between z‐scores = 0.04). In contrast, the neurons that fired the least when rats entered the preferred feeder tended to fire more often during reward trials (p = 2 x 10‐7, mean difference between z‐scores = ‐0.03). The same results were observed when restricting analyses only to sessions in which a significant feeder preference was observed (alpha = 0.05, n = 540, p = 0.002, z‐score diff. = 0.03) in contrast to sessions in which no feeder preference was observed (probability of feeder preference < 0.2; n = 512, p = 0.7, z‐score diff. = ‐0.003). 266 The effect of feeder preference on incorrect‐correct trial activity was futher examined by analyzing only the trials in which the rat entered the feeder the neuron fired most in (as in Figure 10.5). The population of neurons was split into two groups: those neurons that were selective for the rat’s preferred zone, and the neurons selective for the other zones. A dramatic decrease in incorrect vs. correct trial firing was observed when examining only the preferred‐zone selective neurons (p = 0.9, z‐score diff = 0.002). However, there was still a strong difference in the error vs. reward firing when neurons with higher firing on other arms were analyzed (p = 8 x 10‐6, z‐score diff. = 0.04). The hypothesis that increased incorrect versus correct trial activity during cue presentation represents a signal for goal‐directed action was consistent with the observation that these signals increased on sessions in which the rat exhibited a feeder preference. Mysteriously, however, the differential activity between incorrect and correct trials was eliminated when selecting only the neurons that fired most on the preferred feeder zone, and analyzing only the trials in which the rat entered the preferred feeder zone. It did not, however, change the degree of incorrect versus correct firing when rats ran to the non‐preferred feeder. Although these results are difficult to explain, they are not necessarily inconsistent with the hypothesis that the neuron firing is related to goal‐directed action. It might be the case that for preferred zones, the neurons fired just as frequently on error and reward trials, as the rat was relatively more interested in the feeder zone whether or not the rewarded cue was 267 presented from it. For the non‐preferred zones, a transient “interest” or goal was sometimes formed, that was also independent from following the rewarded cue. Chapter summary Prospective activity is an essential ingredient for goal‐directed action selection. In this chapter, the activity across the population of all recorded neurons was examined to evaluate expectation‐based signaling in the mPFC. Principle components analyses revealed that the mPFC was highly sensitive to whether a rat encountered a reward or error at the feeder zone, and that the same neurons exhibited similar differential signaling immediately prior to encountering the error compared with at the reward location. This activity was independent of velocity correlates (which also varied between error and reward trials), and was independent of neural firing increases that accompanied the rats entering the preferred feeder zone on error trials. Although neural activity immediately prior to the rat reaching the feeder zone could be called “expectation,” it is unlikely that activity differences observed at that time contributed to decision‐making. The hypothesis that the cued feeder arms will be selectively represented more than the non‐cued arms when the rats are first presented with cues is not supported by the data; instead only changes in activity related to the arm the rat eventually selected are observed. Interestingly, this activity is higher on error trials compared with rewarded trials. This effect appears to be most prominent in 268 the dorsal anterior cingulate cortex, particularly among burst‐firing neurons. Close examination of the high activity on incorrect trials reveals that is was not caused by an increase in the number of neurons active, which might suggest that the mPFC is considering multiple possibilities, but instead appears to be related to increases in feeder‐arm selective responses, apparently predictive of the neurons’ own activities during the pre‐outcome trial phase. The high incorrect‐trial activity is also modulated by whether the rat exhibits feeder preferences; although it unexpectedly went away when only the neurons selective for the rat’s preferred feeder, and only the trials in which the rat entered the preferred feeder, are examined. This unexpected result may suggest that the mPFC is often represents a particular feeder, whether cued or not cued; though sometimes when the mPFC represents another, non‐preferred feeder, the rat is motivated to run to the other feeder. In general, these data are consistent with the hypothesis that neural responses in the mPFC represent goal locations or target actions, which will be discussed at more length in Chapter 11. 269 CHAPTER 11: DISCUSSION Dissertation overview What is the medial prefrontal cortex, what does it do, and how does it do what it does? The medial prefrontal cortex of the rat (and the anterior cingulate cortices of the primate, which appear to be evolutionary variants of the same thing) contains a collection of neurons of multiple varieties, interacting with one‐another and creating loops with many of the sensory, motor, memory, and emotion systems of the brain (Chapter 2). The region seems to take part in the process by which actions, memories, and emotions are chosen. In contrast with action selection that involves the dorsolateral striatum, the mPFC contribution to decision‐making appears to make use of expectations of the relative advantages between different choices and their outcomes (Chapter 3). And how does it do what it does? The algorithms by which the mPFC functions are constructed by the dynamics of glutamatergic (excitatory) and GABAergic (inhibitory) synaptic signalling, which is in turn modulated by other neurotransmitter systems (Chapter 4). The experiments presented here sought to make a novel contribution to understand these algorithms, by developing a 3‐choice, 2‐cue decision‐ task which rats could perform while neural signals were recorded (Chapter 6). 270 These experiments sought answers to one other question: how do the functional mechanisms of the mPFC change in old age? Evidence suggests that prefrontal systems are particularly sensitive to aging, perhaps in part because of age‐effects on the neuromodulatory nuclei that help determine their activity (Chapter 5). By studying the system under conditions of dysfunction, such as during aging, knowledge is gained about both the aging process and the operations of the system itself. There were four specific hypotheses guiding investigations into the above questions. These were: 1) neural communication would increase during decision times; 2) during decisions, the mPFC would compare different, alternative options; 3) aging would be associated with a reduced ability to overcome previously‐learned cue‐value associations; and 4) aging would be associated with reduced consistency of neural activity during decision‐making. To address these hypotheses, neurophysiological signals were recorded from aged rats as well as young adults while they perform the 3‐ choice, 2‐cue decision task (Chapter 6). More than three‐thousand neurons were recorded, isolated, and analyzed from six aged and six young adult rats performing the decision‐making task. In Chapter 7, behavioral data from the two age groups were presented, revealing several surprising similarities in the abilities of young and aged rats, and two notable differences. The first behavioral pattern that distinguished the aged rats was their reduced speed, the second, their reduced ability to localize auditory cues. The similarities between aged 271 and young adult behavior were reflected by qualitatively similar neural coding, described in Chapter 9. The slower movements and decision‐making in the aged rats was correlated with a slower oscillation frequency of neural communication, the gamma rhythm, also described in detail in Chapter 9. In Chapter 8, neuron classification schemes were presented, and these schemes were applied to examine how the neurons responded to decision‐task demands. This revealed that, on average, the state of the rat cingulate cortex is different when rats are required to attend to the environment and make a choice, as compared to when rats approach their intended goal to discover the outcome. It also revealed a massive degree of variation in the neuron firing properties within the mPFC. Finally, in Chapter 10, an attempt was made to link neuron firing properties with decisions. Factors were identified that contributed to neuron firing patterns related to expectation, others were identified that might contribute to the rat making what appear to be goal‐directed decision. This latter finding was ironic in that extreme efforts were made to minimize the importance of specific goal zones in the task. In this chapter, an attempt will be made to integrate these results into a wider framework, and with the original hypotheses. Because of the diversity in the findings, the discussion will focus on four main topics. The first topic of focus will be to make sense of the behavioral results, the unexpected similarity between young and aged rats and the implications for prefrontal involvement. This section addresses the support, or 272 lack of, for the prediction that aged rats would exhibit different behavior during the decision task (addressing hypotheses 3 and 4 of the list above). The second topic will be a deeper look at age‐related behavioral slowing. Because this topic was not considered in motivating or developing the experiments, it will be useful to place the result within the context of what is known and thought about age‐related behavioral slowing. The third topic will examine the state of the cingulate cortex during the decision task, taking into consideration the increased gamma and theta‐related activity during attention to cues and increased regular‐firing neuron activity when rats were at the feeder zone. The fourth and final discussion topic will evaluate the second experimental hypothesis in the context of the data described in Chapter 10: what activity taking place in the cingulate cortex can be said to reflect variables that are not taking place in rat’s present experience, and how this activity may contribute to the rat preparing and orienting for the future. Interpretation of performance comparison between aged and young rats on an extra‐ dimensional reversal task Behavioral data revealed several surprising findings regarding how rats adapt when the association between reward and a stimulus is switched to a different, independent stimulus. As expected, rats were able to learn the new reward contingencies, although this process took many more trials for both age groups 273 compared with previous results on intradimensional reversal learning tasks (Barense et al., 2001; Schoenbaum et al., 2002; Brushfield et al., 2008) and extra‐dimensional switch tasks (Barense et al., 2001). Another surprise was that rats did not appear to be deliberately correcting their behavior following an incorrect trial. Neither performance, nor reaction times, differed immediately following an error trial as compared to following a correct trial. Further surprises were found when examining the learning differences between age groups. While it at first appeared that aged rats were impaired at adapting their behavior following a reversal to a rewarded auditory cue, closer inspection revealed that this could be accounted for by perceptual differences, as aged rats were generally worse than were young adult at localizing the auditory cue. Also, aged rats learned to follow the visual cue at an equivalent rate to young adult rats. There are several explanations that may account for the apparently slow learning rates across all rats, absence of trial‐to‐trial adjustments, and the absence of an age difference. One notable difference between the present task and reversal‐learning tasks is that the cues are presented together, from the same arm, 33% of the time. Rats therefore continue to be rewarded 33% of the time for following the previously‐correct cue. Second, initial training to localize auditory cues took a relatively long time (see Appendix A) and rats in both age groups generally performed at a lower level in the auditory task in both age groups (Figure 7.1), indicating that the perceptual demands of the task were not trivial. Some learning across days took place even in the single‐cue 274 condition following a switch to the auditory task, which means that adapting to following auditory cues involved more than just learning to ignore the visual cues. There are at least two parallel systems of reinforcement learning, one which integrates the value of stimuli or actions over long periods of time, and one which is thought to be more sensitive to positive and negative feedback in the immediate past. The neural substrates can be dissociated. The long‐term integration system relies at least on loops passing through the dorsal striatum and on D1‐receptor mediated, long‐ term synaptic plasticity; the moment‐to‐moment reinforcement system depending at least on frontal circuits such as the mPFC and orbitofrontal cortex and on the COMT‐ mediated presence of dopamine in synaptic clefts (e.g., Frank and Claus, 2006; Frank et al., 2007; see also the final section of Chapter 3 on dopamine). It is possible that the long‐term integration system of reinforcement learning is at play in the present task, but that the complicating factors discussed above prevent the trial‐to‐trial reinforcement learning system from being noticeably engaged. This possibility is problematic for investigating how the mPFC system reacts and uses recent reinforcement signals in the decision‐making process. It is important to emphasize that this does not mean that the mPFC is not participating in the rat’s choice‐behavior, only that its participation does not appear to rely on reinforcement signals of the immediate past. The similar performance of aged rats on the decision task, with the exception of their poor auditory localization skills and their slower behavior, was matched by the 275 similarity in neural response patterns. The only salient difference in firing properties was observed in those neurons which were sensitive to error and reward expectation, and only when rats performed the more difficult, auditory task. If errors are more common and rats are less reliable at detecting them, then it is perhaps not surprising that error versus reward correlates are less common. In at least one aged rat, neural activity appeared to contain more information about trial phase than was observed in the young adult rats. Since electrode placement within the mPFC was found to be a factor that contributed to information content, the slight differences observed in the location of the electrodes of this rat could not be ruled out as a factor contributing to the difference. However, the rat was motivated and learned to follow the rewarded cue as well as any of the aged animals, and it would be premature to rule‐out the possibility that neural selectivity and performance in the task are unrelated. Why was it that the aged rats had more difficulty localizing sounds than they did localizing visual cues? Two parameters primarily appear to affect an animal’s ability to localize the source of a sound: the bandwidth, or combination of frequencies that the sound is composed of, and the intensity (amplitude, or volume) of the sound (reviewed by Recanzone and Sutter, 2008). An effort was made to increase the bandwidth of the auditory cue by pulsing a 10Hz tone each 50ms, causing the speakers to repeatedly “pop” when initiating the tone and, presumably, increase the richness of the auditory cue (although this was not empirically verified). An effort was also made to adjust the volume of the sound to optimize the rat’s ability to follow the cues. Aged rats still 276 exhibited deficits in following the auditory cues even when the volume was adjusted very high, to just‐below levels that evoked a startle reaction in the rats. The appearance of age‐related deficits in localizing sounds is not a novel finding. The term normally used to describe age‐related hearing declines is ‘presbycusis,’ which can be associated with dysfunctions in peripheral or central auditory systems, and includes deficits in the ability to spatially localize auditory stimuli (Ingham et al., 1997). Brown (1984) found that Spague‐Dawley rats of 21 months and above begin to exhibit deficits in localizing sounds. The peripheral or neural mechanisms of this impairment, however are unknown. The brain’s ability to localize auditory stimuli begins early, in the connection between the cochlear nucleus, which receives input from the ear, and the superior olivary complex (Recanzone and Sutter, 2008). A great deal of experimental and theoretical work has been performed to explain how the superior olivary converts the interaural temporal delays, and the interaural amplitude differences, into a spatial code for where an auditory stimulus originates (relative to the head). Eventually, processed auditory stimuli reaches the superior colliculus, where the egocentric map of where auditory stimuli are coming from can act on motor systems to orient an animal toward or away from a stimulus. In a study aimed at narrowing‐down the neural substrate of age‐related, sound‐localization deficits in rodents, Ingham et al. (1997) recorded multi‐ unit neuronal activity from the superior colligulus of aged guinea pigs exposed to 277 localized auditory stimuli. Results from this study suggested that the spatial map remained stable up until the age of the animals reached approximately 36 months, after which it deteriorated and remained abnormal for the subsequent ten months of age that were tested. Unfortunately, the physiological origin of age‐related sound localization deficits is still unknown. A candidate target of such investigations is the superior olivary complex. However, investigations of the lateral superior olive in aged gerbils (3‐year old)) found no significant differences either in dimensions of the nucleus, or the frequency of inhibitory neurons (GABA and glycine‐immunoreactive neurons) when compared with young adult gerbils (less than 15 months old; Gleich et al., 2004). It is also tempting to identify the aging behavioral deficit with synaptic changes that might take place in old age and affect the ability of the superior olive to integrate interaural time differences. However, the sound localization system is complex and, in mammals, highly plastic; simple models for how interaural differences are detected appear to apply less in mammals than they do in birds (Grothe et al., 2010). In summary, the sound localization deficits in the present study are consistent with previous work, but the nature of this deficit is still elusive. The connection between age related slowing and the speed of the gamma oscillation 278 “Seldom in psychology is a researcher at a loss for new adjectives to describe the reliability of a particular behavioral phenomenon. The phenomenon of a general slowness of behavior with increased age may be one such case” (Salthouse, 1985). Over the past half‐century, investigations into behavioral slowing with age have been in many ways more concerned with identifying exceptions and mechanisms for slowing than for replicating or validating the result. Salthouse himself took advantage of the ubiquity of age‐related slowing to propose a more controversial inference, that the cognitive decline observed with old age in general is a product of slower processing speed (Salthouse, 1996). More specific declines taking place with aging, such as those reviewed in Chapter 5 of this dissertation, emphasize that speed‐of‐processing models of age‐related cognitive changes are an incomplete explanations. On the other hand, they can in many cases explain cognitive deficits, such as those observed in the Stroop task (Verhaeghen and De Meersman, 1998). There is evidence to suggest that both perceptual and decision speed factors are rooted in a single source (Cerella, 1985), a topic that has been more recently reviewed by Salthouse (2000), who also comments on the inability of speed to provide an explanation for all dimensions of cognitive decline. At the very least, speed of processing is among the important dimensions that contribute to age‐related performance changes on perceptual‐motor and cognitive tasks (Birren and Fisher, 1995). It persists even following practice on the tasks, and seems to be only moderately affected by health and individual differences (reviewed by Salthouse 1985, 2000). A neurophysiological correlate of speed changes with aging is a 279 concomitant slowing of event‐related potentials (ERPs; Bashore, 1990). ERPs are scalp‐ recorded electrical potentials that reflect changes of synchronization in neural (neuron or synaptic) summed across a span of the underlying cortex, found by averaging the electro‐encephalogram around a specific behavioral or task event. Bahore (1990) reviews a range of literature demonstrating the slowing of multiple ERP components, prominent among them the P300. Although it is important and interesting that electrical potentials correlate with reaction time speed, it contributes very little to the underlying question: why does the brain slow down with age? A much older literature on aging and slowing attempted to connect reaction time changes with changes in peripheral nerve conduction velocities—it did not take long for this idea to be put aside (Birren and Botwinick, 1955; Salthouse, 1985). More recent examinations have reported changes in conduction velocities in the central nervous system, specifically a change in the distribution of “fast” and “slow” conducting axons in the motor cortex pyramidal tract of aged cats under anesthesia (Liu et al., 1999). These findings can be placed into a larger context of changes observed in central nervous system myelination—in some cases these changes reveal a degeneration of myelin, in other cases increases of myelin (reviewed by Peters, 2002, 2009; Luebke et al., 2010). Myelination changes in the cingulate bundle of aged primates (contrasted with myelination changes in the genu of the corupus callosum) have been associated with cognitive decline (Bowley et al., 2010). The myelination changes in old age also fit into a wide spectrum of results that link age‐dependent cognitive declines with white‐ 280 matter changes in humans. These are evaluated using structural, magnetic resonance imaging (MRI) including the diffusion tensor imaging (DTI) technique that articulates white matter tracts. One recent study claims to have identified a specific factor that combines information on white matter fractional anisotropy, mean diffusivity, and radial and axial diffusivity to explain 45% of individual differences in processing speed (Penke et al., 2010). Perhaps independent from axon conduction changes, recent findings have observed improvements of age deficits using pharmacological manipulations. Notable among these are manipulations of the GABAergic system (e.g., Leventhal et al., 2003; Lasarge et al., 2009). Both of these results have shown that changes to the GABAergic system can lead to improvements in perceptual tasks. Leventhal et al., (2003) demonstrated this by administering GABA, or the GABA agonist muscimol, to improve monkey visual performance. Interestingly, in this study administration of GABA also “turned on” the coding properties of the neurons, endowing them with a previously absent tuning selectivity. While this could be interpreted as a “fix” of a broken GABAergic system, it seems also possible that the pharmacological manipulation changed the degree to which the monkey was engaging its visual system. In the Lasarge et al. (2009) study, GABA(B) antagonists were found to improve olfactory discrimination learning deficits in cognitively‐impaired rats. 281 In Chapter 9, age‐dependent slowing was found to correlate with changes in the frequency of the gamma oscillation. Chapter 4 reviews evidence that the gamma oscillation is important for neural communication—synchronization of neural activity within gamma stabilizes the circuits that are “talking” to one‐another, putatively creating dynamic, attractor‐like assemblies. If there is an increase in the space between each step of neural communication, as would be likely to happen with reduced frequency of the gamma oscillation, then it should take longer, overall, to complete all steps of processing necessary for a particular response, behavior, or cognitive process (Figure 11.1). This concept is the essence of Cerella’s, 1990 review of processing speed with aging; although Cerella proposed that processing speed changes not because each step is necessarily slower, but because information degradation in aged networks require more processing steps to take place. While this mechanism may explain age‐ dependent slowing under some conditions, the current results on the similarity between coding properties of cortical cells in aged and young adult rats would suggest that, in fact, it is the speed of communication between sets of neurons that suffers. 282 Figure 11.1 Illustration of hypothesized slowing of processing speed resulting from slower gamma oscillations. An illustrated depiction of faster, young adult (top) and slower, aged (bottom) gamma oscillations (black wave trace). These oscillations represent waves of increased and decreased neuronal and synaptic activity. When the neural activity reaches a threshold (described at an arbitrarily‐chosen level by the horizontal dashed lines), a sufficient amount of input activity will evoke a spike in a post‐ synaptic neuron. The time constant over which the neuron can integrate its inputs is depicted by the horizontal blue lines, placed above the gamma oscillation peaks to describe the integration period when a sufficient number of presynaptic inputs are active. Over the same period of time, aged rats will theoretically have fewer gamma peaks over which to evoke responses in a post‐synaptic population (which in turn would be expected to feedback onto the presynaptic population, causing a synchronous, dynamic assembly). Although the figure equates processing speed with the evoked activity of a post‐synaptic neuron, it is not necessarily the firing rates of post‐synaptic neurons, but rather the formation of the dynamic assemblies, that gamma might be contributing to. The relationship between gamma speed and speed of processing is further supported by the known effects of barbituates. These GABAergic agonists affect the GABA receptor time constant, and in doing so decrease the frequency of gamma (Whittington et al., 1995). Barbituates also, meanwhile slow reaction times (Meador et al., 1995), and seem likely to have a general effect on processing speed. 283 The data presented in Chapter 9, as with previous reports (Czesivari et al., 2003) suggest that the gamma frequency is a product of both the activity of fast‐spiking inhibitory neurons, and the stimulation of these inhibitory neurons by local excitatory cells. Based on the timing of excitatory neuron firing relative to inhibitory neurons, and with reference to the gamma oscillation, the age‐related change to this circuit appears not to be a change of inhibitory influences on excitatation, but in the degree to which the excitatory neurons are driving inhibition. Consistent with this possibility, a recent report has demonstrated that gamma frequency is modulated by glutamatergic action on NMDA receptors located on the inhibitory cells (Mann and Mody, 2010). Future studies are warranted to examine the effect of specific pharmacological manipulations on both the gamma speed and behavioral speed. Distinct network states in the cingulate between attention/decision and reward‐ acquisition epochs Neurons were classified into putative excitatory and inhibitory groups, and independently classified into fast‐spiking, burst‐firing, regular firing, and theta‐firing cells. At millisecond time‐scales, there was a correlation between the firing of excitatory and inhibitory neurons: excitatory neurons tended to fire just after gamma peaks, while inhibitory neurons tended to fire several milliseconds later. This relationship is likely to be what underlies the gamma rhythm, which itself reflects the 284 waxing of synaptic activity, presumably from local excitatory neurons, kept phasically in check by synaptically‐connected inhibitory neurons. At wider time scales, the order of seconds, there was an apparent anticorrelation between excitatory and inhibitory activity. When cues were presented to the rat upon the rat’s entry into the cue zone, there was an average increase in the activity of inhibitory neurons (many of them fast‐ spiking neurons firing at the pace of gamma) and of theta‐firing neurons. Not surprisingly, accompanying this increase was an observed increase in the average power of gamma. As the rat approached the feeder zone, however, and particularly as the rat received reward, the average activity of putative inhibitory neurons and neurons bursting at theta frequency (called theta‐firing neurons), as well as the power of the gamma oscillation, was reduced, while the average activity of regular‐firing neurons, including the number of active regular‐firing neurons, increased. As reviewed in Chapter 4, in primary sensory regions gamma power normally increases when input signals to the region increase, and gamma events in the hippocampus have been associated with increased input from region CA3 (Colgin et al., 2009). It can be inferred that increased inhibition and gamma‐related activity during the decision period is representative of increased input, perhaps from the thalamus or cortex, and correspondingly increased communication between the cortical regions. It is possible that some kind of tradeoff exists, where reduced cortical communication means increased activity relying on local network synapses. Alternatively, there could be a tradeoff between communication between cortical regions and communication 285 with non‐cortical regions, such as the brainstem or hypothalamus, anatomical loops that may be highly engaged during the processing of reward (or error) and its expectation. What is particularly striking about these data is that the highest number of neurons seemed to be active and selective for the period of time with the least amount of inhibition and local oscillation. Decisions and the medial prefrontal cortex: the importance of expectation in guiding behavior This dissertation began with a discussion about decisions, and with a rally to hunt for the demons that pull the levers of our behavior so that we might learn what they are and how they interact. The results of the experiments described in this thesis, however, has been a trudge through physiological details of the mPFC during performance of decision tasks. Did this long process pay‐off in uncovering something about what makes a decision? Chapter 3 revealed that “expectation” is likely to play a fundamental role in mPFC functioning. Although the 3‐choice, 2‐cue task was designed to assess decision‐ making, there is no reason to believe that expectation was required for task performance. Over days of performing the task, rats were likely to develop value associations with the presented cues, because of the contingencies between the stimuli and reward. Following a switch in which of the two cues was rewarded, the newly 286 rewarded cue would become a secondary reinforcer, evoking orientation and approach behaviors, while the formerly rewarded cue gradually lost its positive valence and ceased to stimulate these Pavlovian responses. However, just because the mPFC may not be necessary for task performance (or, arguably, for task learning: see Chapter 7), this does not mean that the mPFC has “turned off” during task performance. In fact, many neural correlates of movement, space, and outcome were observed in the mPFC while rats performed the task, and the hypothesis was addressed that these neural patterns played some role in determining the rat’s behavior. Identifying relationships between the neural activity and the rat behavior often meant looking for neuron activity that was predictive of experiences the rat had not yet encountered. This included examination of the degree to which rats predicted an error before the rat encountered the error sound at the end of the feeder arm. It also included feeder‐arm selective activity that apparently preceded the rat making its choice by entering the feeder arm. These findings have implications for theories of how the mPFC functions during decision‐making, which are presently discussed. Expectation is likely to be important for mPFC function for the obvious reason that expectation of future outcomes informs how a rat should decide between possible actions or prepare itself to behave in the future. This idea led to the hypothesis that when a rat was evaluating its choices, such as during the initial cue‐presentation in the 3‐choice, 2‐cue task, the actions and locations associated with the location of the two cues would be active in the mPFC, while those associated with the non‐cued feeder 287 would not be. This hypothesis was not confirmed by the data. But it is possible that the non‐cued feeder was just as frequently considered as a viable option to the rat as the cued feeders, in which case the hypothesis would still be valid, but assumptions about how the rat selected among the possibilities (the assumption that the rat considered only the 2‐cues, not the 3‐choices) were false. In fact the data revealed that many feeder‐zone selective neurons increase activity on error trials, suggesting that perhaps more neurons were active during error trials because the rat was considering between its many options (“considering” in this context means that the expectations associated with each outcome, rather than just a single outcome, would be represented during the same 500ms window). However, restricting analyses to only trials in which the rat entered the zone “coded for” by each neuron revealed that the effect was less likely due to increases in activity of all alternative options, and more likely the result of increased activity within the neurons coding the choice the rat was in the process of making. These results were unexpected for many reasons. They were inconsistent with the hypotheses because they did not seem to fit a model that claims decisions are made by comparing the expected outcomes of those decisions. They were furthermore unexpected because they do not seem to fit with findings from the human dorsal medial prefrontal cortex. Based on convergent data, Young et al. (2004) hypothesized that many physiological patterns observed during decision conflict and errors, localized to the anterior cingulate cortex, were related to the representation of the multiple action sets, including the non‐performed set that becomes compared with the performed 288 action when a subject executes an error (see Chapter 4 section titled “Relationship between cognitive control and action selection”). Modirrousta and Fellows (2008a) further found that patients with damage to the dorsal anterior cingulate cortex were impaired at rapidly correcting actions that the recognized were errors—but not impaired at identifying verbally or by an action that they had made an error. This finding also supports the notion that the dorsal anterior cingulate is capable of representing multiple actions, so that when the error is committed, the simultaneously maintained action is prepared and the error is quickly corrected. Is the absence of simultaneous activation of different motor sets in the present data strong enough, and can it be sufficiently generalized, to reject the hypotheses built from observation of the human cingulate? Probably not, on account of a number of factors that might include species differences, potential differences in the region of examination, and the specific task the animals were required to perform. However, the data likely do have important implications for the mechanisms by which the medial prefrontal cortex contributes to decision‐making. Rather than evidence of a comparison being made between possible actions or destinations, the increased activity observed during decision making, on error trials, appeared to be related to the prospective activation of a particular reward zone that the rat subsequently selected. This activity also appeared to increase on sessions in which the rat exhibited a significant feeder preference, futher suggesting the possibility that the activity represented an action‐value or destination‐value signal that possibly 289 directed the rat’s selection to one particular feeder zone. If this is the case, then the reason why mPFC activity (particularly activity among burst‐firing neurons in the anterior cingulate cortex) increased before error commission is because the mPFC was processing separate information from the systems that contributed to the rat’s performance on the task itself. That is, the rat’s ability to follow the high‐valued cue might have been dependent on a system entirely separate from the dorsal mPFC. This theory is supported by the hypothesis, and by evidence presented in Chapter 7, that neither expectation nor the dorsal medial prefrontal cortex are required for accurate performance in the present task. The increase in excitatory activity during cue‐following might therefore represent the mPFC “weighing‐in” on the decision‐process, directing the rat to a choice that has a temporary value due to factors unrelated to the cue itself. This could possibly include values associated with exploration of feeders not recently visited, or a perceived pattern in the sequence that the cue was presented in. Thus, the increased activity during decision making on error trials provides the first clue of the mechanisms by which the mPFC contributes to goal‐directed action. Expectation responses were observed in the mPFC at other trial phases of the task, which likely did not contribute directly to the decision‐behavior, but may have influenced movement and movement preparation. For example, many neurons were identified that exhibited differential firing between when a rat was reaching the feeder zone on an incorrect trial compared with when the rat approached the feeder zone on a correct trial. 290 Some of the neurons that fired differentially between error and reward trials were not related to expectation per se, but rather appeared to fire differently because the rat moved more slowly on error trials. This was established by comparing the velocity‐responses at one trial phase, the pre‐outcome period, with other trial phases, such as the post‐outcome period (during which rats tended to run more quickly following an error). Interestingly, many of the neurons increased firing rates when the rat was moving more slowly. This does not make intuitive sense: in theory, the rat is engaging less energy and encountering less sensory flow when it moves more slowly. An intriguing possibility, and one that fits with the cognitive control literature, is that these mPFC neurons actually play a role in slowing the rat’s running speed, for example during times when the rat is unsure about whether it is making an error. Alternatively it could be that the mPFC is generally more engaged when the rest of cortex is not bombarding it with other signals (a theory that would be consistent with the observed tradeoff between inhibition during initial cue presentation and increased excitatory activity when the rat is sitting quietly at the feeder, discussed in the previous section of this chapter). Neuron population activity differences between incorrect and correct trials were not strictly due to velocity differences, however. This could be determined by applying correlation analyses or principle components analysis over a wide range of factors, including error‐versus‐reward trials during different trial phases. These analyses revealed that some of the differential activity in the neuron population related to the 291 rat encountering an error or a reward was preceded by similar patterns differentiating the two outcomes prior to the presentation of the outcomes. These neurons did not necessarily predict the outcomes themselves, but may have been involved in preparing the rat for the different actions associated with the two different outcomes. Further work will be necessary to reveal whether the apparent preparedness of the rat for the outcome was related to the expectation‐related activity. Further work will also be necessary to determine whether there is a relationship between neurons signaling the possibility of an upcoming error, and those neurons firing in relation to decreases in the rat’s running speed as it reaches the feeder zone. The neuron activity putatively involved in “goal‐directed action” is not necessarily unrelated to the neuron activity associated with the expectation of a reward or error. In the present task, the feeder‐arm selective neural activity increases observed on error trials may have represented a signal that led rats to approach a particular feeder. Expectation of rewards immediately preceeding the rat entering the feeder zone may also have contributed to approach behaviors; while, conversely, expectation of error as rats entered the feeder zone may have contributed to the slowing‐down and even withdraw of rats from the feeder zone they had selected. Thus, in both cases the “prospective” activity can be re‐fit in the context of approach and withdraw. Chapter 2 described the dorsal medial prefrontal projections to the superior colliculus, an region that might be involved in a rat orienting to, or possibly even withrdrawing from, a particular stimulus. Further work will be necessary to reveal how the cingulate‐collicular 292 projections are related to approach to a particular location apparently associated with a high‐value outcome. One other topic still left without full explanation is how these prospective signals are generated: how does the local network cope with incoming information to elicit the expectation reponses, and how do local computations process the signals. Although the burst‐firing excitatory neurons may play a particular role in representing goal location, there was no obvious antagonistic relationship, or correlation, observed between these and the inhibitory neurons firing during the same trial phase. Further examination and modeling of the circuits will be necessary to combine the coding principles described in this thesis with the anatomical loops discussed in Chapter 2, the precise contribution of the mPFC to decision‐making described in Chapter 3, and the nature of the neural interactions (Chapter 4) that lead to the goal‐representations and expectation signals. 293 Appendix A: Materials and Methods Subjects All experimental procedures were in accordance with NIH guidelines. Aged (24‐ 31 months) and young adult (9‐14 month) Fischer‐344 rats were first tested on the Morris Water Task for spatial and visual performance abilities. Rats unable to perform the visual portion of the water maze, which required using a salient visual stimulus to find a hidden platform, were removed from the experiment. Animals were then food restricted until they were motivated to run for liquid food reward, vanilla Ensure (Abbott Laboratories; Abbot Park, Illinois; normally 80‐90% of free feeding weight). An additional group of rats, roughly 1/3 of the remaining aged animals, were removed from the experiment on account of deficits in localizing auditory stimuli (see below). A total of 11 aged and 10 young adult rats were studied in the behavioral task, and electrophysiological recordings were collected from 6 aged and 6 young adult rats. Experimental apparatus The experimental apparatus was a platform with three arms radiating from a circular, central region (see Figure 6.1 A and B). Experimental control software was custom‐written in Visual Basic 6 (Microsoft; Redmond, WA) and Basic‐X (NetMedia Inc; Tucson, AZ), and allowed position data collected from an overhead camera to be 294 interfaced with the Neuralynx 32‐bit Analogue Cheetah Data Acquisition System (Neuralynx; Bozeman, MT). At the end of each arm of the platform was an auditory speaker, a light emitting diode (LED), and a solenoid‐based, liquid‐food feeder. The system of experimental control and recordings is also presented in Figure A.1. Training and Task Tracking: Real‐time position of the rat was required for the experiment to operate, as it initiated specific task events. In the rats implanted with “hyperdrive” electrode recording probes, position could be determined by the LEDs located on the headstages (described below). In the absence of an implanted electrode drive, as in most behavior recordings and during training, a custom‐designed jacket was used. Jackets consisted of a narrow tube of stretchy fabric with two holes for the rat forearms, a thin watch battery, a red LED, a resistor, a wire, a scrap of cardboard or plastic, and a small binder clip. Training: Rats were trained in the following sequence (name of task and approximate number of session rats required in parentheses): 1. (Pretraining 1, 1‐2 days) Rats were allowed to move freely on the platform while feeders were baited with vanilla Ensure. If the rats consumed the drops of Ensure at a feeder, the drops were replaced once the rat returned to the central zone. 295 2. (Pretraining 2, 2‐4 days) A 500‐1000ms auditory cue was triggered immediately before the rat reached a baited feeder, so that the rat would associate the cue with reward. Often a feeder was left unbaited, in which case no auditory cue was triggered. 3. (Supplemental training, variable time) In cases where interfering factors or training errors reduced rats’ motivation to zero (rats gave‐up running), rats were led to the ends of feeders by hand, with a plastic weighboat of Ensure and an auditory cue (specs. below) that repeated for as long as the rat was moving in the correct direction. 4. (Auditory Localization task, 5‐14 days) A 3‐second auditory cue was triggered when rats reached the central zone of the platform. If rats correctly reached the end of the cued arm, they were rewarded with a drop of Ensure 5. (Visual Localization task, 3‐5 days) Once rats could perform above 65% on the Auditory Localization task, the sound cue was replaced by a visual cue. During the initial few days, the visual cue was presented for 10 seconds or until the rat reached the end of a platform arm. The cue interval was then reduced to 3 seconds. Rats recorded from electrophysiologically, who underwent surgeries (described in the next section), were first trained on the task, then placed on full food in the days prior to and at least four days following surgery. They were then food restricted and 296 reintroduced to the task, retrained for two to three additional weeks before recording sessions began (concurrent with the lowering and stabilizing of electrodes; see below). Cues: The auditory cue was a localized ringing sound generated by repeating a 50 ms, 10 kHz tone between 25ms delays, the visual cue was a white LED that blinked at 4Hz. Sessions: The sequence of sessions in both the behavioral task and electrophysiological recording task is described in Figure 6.2C. Once rats were trained on the auditory and visual localization tasks, they were presented with the two‐cue task. Following 8 days of performing the 2‐cue task for at least half of one session each day, the rewarded cue was reversed in the middle of the session. The same cue remained the rewarded cue for the subsequent 7 days, followed by a day when the cues were once again reversed. Rats continued on this schedule for a number of switches. In the behavioral study, all rats completed at least 15 sessions (nearly two switches), with eight pairs of rats completing at least 33 sessions (4 switches). Feeders: At the beginning of each session, feeders were primed so that the liquid Ensure reached the tip of each feeder tube. Three drops of Ensure (normally 130ms solenoid release time per drop) were deposited from each feeder into a small weigh boat and weighed to determine relative differences between feeder release. Each drop of room‐ temperature Ensure weighed between 0.67 ‐1.3 g; if any feeder was releasing more or less than ~0.25g /drop compared with the others, the feeders were manually cleaned and the process was repeated until the feeders were even. The occurrence of clogged 297 feeders was extremely rare, only taking place in the very early months of the experiment before a systematic feeder cleaning routine was decided upon. Additionally, experimenters kept a close watch on the rat behavior, and if the rat began spending less time at a feeder, the rat was placed in the rest chamber (a flowerpot) and the problem was resolved. Recording probes and surgeries Rats recorded from electophysiologically were implanted with a device called a “hyperdrive” that contained 14 “tetrodes”, where a tetrode is an electrode made of 4 twisted wires (0.0005’ diameter, 3305 Ohms/ft). Each wire of 12 of the tetrodes (48 wires) were connected to independent channels of a circuit‐board, that could, in turn, interface with a headstage and Cheetah amplifier system (see below). The two remaining tetrodes were used as reference electrodes, and were connected each to a single channel of the circuit board. Implantation surgeries in rats recorded electrophysiologically were performed under isofluorane anesthesia. The hyperdrive was implanted to the rat’s skull, over a craniotomy centered at 1.3mm to the right of midline, tilted 9o medially, and anterior to bregma 2.4‐3.2mm. This position allowed the tetrode recording probes to be lowered into position (below). During recording sessions, one of the two reference electrodes was lowered only to the surface of cortex, and the other was lowered to the prelimbic region. Each 298 tetrode within the hyperdrive could be lowered independently by turning a screw, causing the tetrode to pass through a cannula (28 gauge inner‐diameter, 30 gauge outer‐diamter) within a bundle of 14 cannulae, soldered together at one end. Tetrode depths were kept track of by the number of turns of the screw (where each full turn pushed the electrode down roughly one‐third of a milimeter). Rats were additionally implanted with local field potential (LFP) electrodes targeted to the hippocampal CA1 later. Each LFP electrode was a combination of two electrodes roughly 500µm apart (so that the second electrode targeted the hippocampal fissure). Normally, wires of 0.003’, 0.004’ diameter were used, and positioned together with 22 gauge tin wire (using heat‐shrink tubing and thin super glue), which provided a handle for stereotaxic implantation. Hippocampal LFP electrodes were implanted to the same side of the brain as the mPFC hyperdrive. In general, analyses examined the hippocampal electrode on each rat that exhibited maximal theta oscillation. In most rats, an EMG electrode was also placed laterally to one side of the rat’s neck; these signals were recorded by not analyzed in the present dissertation. All LFP and EMG wires were connected to the circuit board on the hyperdrive device, to be interfaced with the Cheetah recording equipment (below). Experimental control and recordings 299 The experimental control and recording system are presented in Figure A.1. All recordings were collected using a Neuralynx 32‐bit Analogue Cheetah Data Acquisition System (Neuralynx; Bozeman, MT). Electrodes interfaced with amplifiers through a headstage, cables, and commutator that allowed the rat free movement and rotation around the platform. Continuously recorded LFPs were sampled at 1989Hz (2 aged, 3 young) or 1659Hz (4 aged, 3 young) and filtered at 1‐475Hz. Action potentials were detected online by thresholding 32kHz sampled, 600‐6000Hz filtered signals. Spikes were sorted offline as originating from independent single neurons according to clusters of waveform shapes and features across the four channels of a tetrode. Spike data was sorted into independent neurons using a combination of the open‐source software Klustakwik (http://klustakwik.sourceforge.net) and MClust (http://redishlab.neuroscience.umn.edu/MClust/MClust.html). Video tracker data was collected at 60Hz from an overhead camera, behavioral events were recorded directly by parallel outputs from the Basic‐X into the Cheetah acquisition system. Cluster cutting guidelines used in the present study included the following: 1) Clusters from Klustakwik are sorted ‐‐non‐biological clusters are removed from the pool, and the rest are color‐coded according to candidate clusters to merge. 2) Clusters with spikes between 0‐5ms are trimmed using a waveform cutter and in some cases standard waveform principle components. 3) Cross correlations are examined for candidate clusters to be merged, particularly those color‐coded the same in Klustakwik. 4) When all clusters are finished, energy projections are quickly examined to verify no cluster is 300 clearly overlapping with another and should be merged. 5) All clusters are written to file, and are later sorted through to determine if the quality is high enough to use for analyses. Computer: interface for recordings and experiment control Cheetah recording hardware Cheetah Recording Cables Amplifiers Basic‐X: controller card converting serial commands to 5V ttl pulses Ceiling Camera Optical isolator 2 nd Basic X 3‐arm platform 555 timer circuit Figure A.1 System of experimental control and recording. Signals from the computer controlling the task (red arrows) also required information about the rat (black arrows). Specifically, rat movement data was collected by a ceiling camera (shown in the upper right side), passed through the Neuralynx Cheetah data acquisition hardware, and was read by a visual basic program on the recording and control computer (shown in the upper left). The visual basic software controlled outputs through a serial port to the Basic‐X controller card, which in turn delivered outputs to optical isolators controlling the feeders, a second Basic‐X card that controlled the sounds, a 555 timer circuit that controlled the blink frequency of the LED lights, and information about its output to the Cheetah data acquisition system. In the recording task, electrophysiological signals also were carried by cables, through a commutator that allowed the rat free rotation, and to the Cheetah amplifiers. 301 Analyses All analyses were custom written in Matlab (Mathworks; Natick, MA). Behavior: Output from the Basic‐X to Cheetah signaled event timestamps, these were processed offline in the calculation of trial completion time and error rates. Video data was smoothed with a 20‐step (~330ms) hamming window, and instantaneous velocity was calculated by the position change between 33ms time steps. Instantaneous velocity was used to identify the time the rat made a decision in each trial (the acceleration point to the chosen feeder), and the initiation of the rat’s return from the feeders (the acceleration point following the low‐velocity period at the feeder). LFP: Data was analyzed 1) raw, 2) filtered with a whitening filter (2nd‐order Yule‐Walker autoregressive coefficients, averaged across non‐licking task periods), or 3) filtered with a 40‐70Hz bandpass filter (4th‐order Chebyshev Type 1). The primary measure used to compare frequency between aged and young adult rats (Chapter 9) was peak gamma frequency, the frequency that exhibited relatively highest power. In unwhitened data, the 1/F factor made it difficult to interpret peaks under 40Hz, and it was likely that unwhitened peak frequencies are shifted low as a result of 1/F spillover. Since higher power is found in lower frequency signals (the 1/F component), use of a whitening filter was valuable in identifying relatively strong frequencies. In whitened data, peaks were searched for above 20Hz. Peaks between 59‐61Hz were not considered to avoid artifact from 60‐cycle alternating current in the building (sampling rates and selected time interval limited the spread of this artifact to within this range, confirmed by a narrow 302 spike in power observed only between 59‐61Hz in all rats). To identify peak‐frequency during rest periods, only frequencies above 40Hz were examined, since gamma power was relatively lower than lower‐frequency oscillations that may have accompanied sleep. To measure relative changes in power, such as between trial phases, LFPs were bandpass filtered at 40‐70Hz and instantaneous power was calculated by the absolute value of the Hilbert transform, normalized by means across non‐licking periods (which helped control for impedance differences across electrodes). High‐voltage spindles were identified as increases in 7‐9Hz power greater than 2 standard deviations of mean power. LFP methodological considerations: In most rats, local field potentials (LFPs) in the mPFC were recorded by finding the voltage between an individual tetrode compared with a reference tetrode placed near the surface of the cortex. Hippocampal LFPs were recorded simultaneously, by recording the voltage from two electrodes targeted to the hippocampal layer and fissure compared with the voltage of the cannula bundle placed against the surface of the cortex. In two younger rats, a cerebellar screw was additionally used as a reference. One methodological constraint in examining the LFP is the appearance of two kinds of artifact: 1) electrical or mechanical artifact when rats were licking made it impossible to record LFP during periods during reward periods 2) low‐frequency changes in the tetrode‐recorded LFPs (i.e., the mPFC LFPs) were often present when the rat quickly accelerated either linearly or rotationally, making examination of low‐frequency changes in the LFP during task performance suspect. This 303 latter constraint was prohibitive in evaluating unfiltered event‐related potentials (ERPs), since the movement‐related mechanical artifact disguised more subtle, biological signal changes in connection to specific task or behavioral events. In the examination of oscillations, natural signals exhibit a “1/F” component, which means that, because of the dynamics of spatio‐temporal summation, lower‐frequency components of a signal always carry larger magnitude changes than high‐frequency. Analyzing signals can therefore be often aided by a whitening filter, which circumvents the 1/F component by using an autoregressive filter. This is sometimes implemented in the present analyses (see also Buzsaki, 2007; Sirota et al., 2006 supplementary materials). LFP‐Single Neuron: A 40‐70Hz bandpass filter was also used when identifying the phase of firing of single neurons. Both positive and negative zero‐crossings were identified in the filtered LFP, and spike phase was inferred based on where it fell between two positive and two negative zero crossings. Spike triggered‐averages (STAs; averages of a window of the local field potential centered on each spike) were generated from 20‐ 200Hz filtered LFPs to remove low‐frequency components. Analyzes on STA gamma period length, and inhibitory versus excitatory gamma phase relationships, a 40‐70Hz filter was used to remove peaks from higher‐frequency components that were not the subject of investigation. Single neuron classifications: Classification of neurons as excitatory and inhibitory used many of the same methods as Bartho et al. 2003. Cross correlations were first computed between all neurons with > 1Hz firing rates during the task. If cross‐ 304 correlations exhibited a dip lower than 3 standard deviations from baseline within 5ms after the alignment spikes, for more than one bin, the neuron was a candidate inhibitory neuron. Consistent with previous reports, candidate inhibitory neurons tended to have narrower half‐amplitude times and peak‐to‐trough times (plotted in the “negative‐up” convention; Fig S3A). All neurons in this narrow waveform grouping were classified as putative inhibitory neurons. Neurons in the much larger grouping were classified as putative excitatory neurons. An additional grouping of exceptionally narrow waveforms were found to also have very deep preceeding hyperpolarizations, these were labeled as axon fibers and not used for the present analyses. Neuron groupings based on firing characteristics used two features derived from the autocorrelations: latency to peak and the slope of decay. Autocorrelations were generated with 1ms bins. Decay slope was calculated by first smoothing the autocorrelation with a 10‐step (10ms) hamming window, then identifying the x‐y location of the peak and x‐y location of the first bin in which the autocorrelation reached half of baseline (where mean autocorrelation values between 40‐50ms was used as baseline). Theta‐firing neurons: Theta‐firing neurons were identified by the autocorrelations. Associating neurons with a theta‐firing index began by creating a spectrogram from the autocorrelation of each neuron and selecting out those which showed peaks between 6‐10 Hz and had an average firing rate > 0.5 Hz. This yielded ~47 neurons that visual inspection of the autocorrelations confirmed were highly modulated by theta (theta‐firing neurons). The mean autocorrelation of these neurons exhibited peaks between 90‐140ms, 40‐90ms, and 210‐260ms and troughs 305 between 40‐90ms, 140‐210ms, and 260‐330ms. Theta firing indices were calculated as the addition of the ratio for each peak and its preceding trough (peak1/trough1 + peak2/trough2 + peak3/trough3). This yielded a distribution centered at 3 and with a standard deviation of 1.1. One concern with this method is the possibility that theta frequency varied with the variables examined, and that differences in theta firing power could be attributed to shifts in the location of the peaks. This could be addressed by referring back to the original set of thete‐firing neurons and identifying whether the variables of interest varied in this set, which did not appear to be the case. Auto‐ and Cross‐correlations: All auto‐ and cross‐correlations used 1ms bin sizes. Baseline mean and variance in cross correlations were calculated from ±10‐50ms periods. Binned single neuron activity: Most analyses on single neurons were performed after binning the neural activity according to trial phase. The trial phase bins included the following: 1‐‐initiation of return to the cue zone, 2‐‐middle of return, 3‐‐immediately prior to task cue presentation, 4‐‐immediately following initiation of cue presentation, 5‐‐around the decision time, 6‐‐immediately prior to the outcome, 7‐‐immediately following the outcome, 8‐‐reward sampling, 9‐‐response to the error sound). In many cases, binned data was converted into z‐scores by subtracting the average firing rate across all bins from all trials and dividing by the standard deviation across all bins from all trials. 306 Appendix B: Histology Electrode drive implants centered the tetrdoes 1.3mm to the right of midline, tilted 9o medially, and anterior to bregma 2.4‐3.2mm. They were then lowered progressively over the course of several months, so that recordings could be collected from medial precentral (500‐1800 µm), dorsal anterior cingulate (1800‐2900µm) dorsal prelimbic (2900‐3600µm) and ventral prelimbic (3600‐3900 µm)regions (see also Figure 8.4B). When rats finished the recording protocols, the electrodes were placed at 3000µm, or in several rats, at 2000µm and 3000µm, and lesions were made by passing 5µA through electrode tips for 10 seconds each, and rats were left for 2‐5 days before perfusion. Figure B.1 Location of histological lesions in young and aged rats. Left: microscope images of coronal sections with lesion taken from one old‐young pair. Right: each red point represents the identified location of a lesion in a young rat, each blue point 307 represents the position of electrode lesions in aged rats. Lesion locations were highly overlapping between the two age groups. Figure B.2 Estimates of actual electrode position from tetrode turn depths and histology. In each rat, the electrode lesions were estimated and compared with the tetrode depths. In this coronal section, lesions were found slightly more medially and dorsally than what was predicted by surgical placement of the drive and depths of the electrodes (roughly 200µm more dorsal). 308 REFERENCES Akert, K. (1964). [Neurophysiological Observations on the Problem of the Topography of Skin Diseases.]. Arch Klin Exp Dermatol, 219, 54‐63. Allard, S., Gosein, V., Cuello, A. C., & Ribeiro‐da‐Silva, A. (2010). Changes with aging in the dopaminergic and noradrenergic innervation of rat neocortex. Neurobiol Aging. Alonso, A., & Llinas, R. R. (1989). Subthreshold Na+‐dependent theta‐like rhythmicity in stellate cells of entorhinal cortex layer II. Nature, 342(6246), 175‐177. Amano, T., Unal, C. T., & Pare, D. (2010). Synaptic correlates of fear extinction in the amygdala. Nat Neurosci, 13(4), 489‐494. Amat, J., Baratta, M. V., Paul, E., Bland, S. T., Watkins, L. R., & Maier, S. F. (2005). Medial prefrontal cortex determines how stressor controllability affects behavior and dorsal raphe nucleus. Nat Neurosci, 8(3), 365‐371. Amiez, C., Joseph, J. P., & Procyk, E. (2006). Reward encoding in the monkey anterior cingulate cortex. Cereb Cortex, 16(7), 1040‐1055. Anagnostaras, S. G., Maren, S., & Fanselow, M. S. (1999). Temporally graded retrograde amnesia of contextual fear after hippocampal damage in rats: within‐subjects examination. J Neurosci, 19(3), 1106‐1114. Arnsten, A. F. (1998). Catecholamine modulation of prefrontal cortical cognitive function. Trends in Cognitive Science, 2(11), 436‐447. Arnsten, A. F., & Goldman‐Rakic, P. S. (1984). Selective prefrontal cortical projections to the region of the locus coeruleus and raphe nuclei in the rhesus monkey. Brain Res, 306(1‐2), 9‐18. Atallah, B. V., & Scanziani, M. (2009). Instantaneous modulation of gamma oscillation frequency by balancing excitation with inhibition. Neuron, 62(4), 566‐577. Baddeley, A. (2003). Working memory: looking back and looking forward. Nat Rev Neurosci, 4(10), 829‐839. 309 Baeg, E. H., Kim, Y. B., Huh, K., Mook‐Jung, I., Kim, H. T., & Jung, M. W. (2003). Dynamics of population code for working memory in the prefrontal cortex. Neuron, 40(1), 177‐188. Balleine, B. W., & O'Doherty, J. P. (2010). Human and rodent homologies in action control: corticostriatal determinants of goal‐directed and habitual action. Neuropsychopharmacology, 35(1), 48‐69. Barak, O., Tsodyks, M., & Romo, R. (2010). Neuronal population coding of parametric working memory. J Neurosci, 30(28), 9424‐9430. Barense, M. D., Fox, M. T., & Baxter, M. G. (2002). Aged rats are impaired on an attentional set‐shifting task sensitive to medial frontal cortex damage in young rats. Learn Mem, 9(4), 191‐201. Bartho, P., Hirase, H., Monconduit, L., Zugaro, M., Harris, K. D., & Buzsaki, G. (2004). Characterization of neocortical principal cells and interneurons by network interactions and extracellular features. J Neurophysiol, 92(1), 600‐608. Bartos, M., Vida, I., & Jonas, P. (2007). Synaptic mechanisms of synchronized gamma oscillations in inhibitory interneuron networks. Nat Rev Neurosci, 8(1), 45‐56. Batuev, A. S., Kursina, N. P., & Shutov, A. P. (1990). Unit activity of the medial wall of the frontal cortex during delayed performance in rats. Behav Brain Res, 41(2), 95‐ 102. Baxter, M. G., Hadfield, W. S., & Murray, E. A. (1999). Rhinal cortex lesions produce mild deficits in visual discrimination learning for an auditory secondary reinforcer in rhesus monkeys. Behav Neurosci, 113(2), 243‐252. Baxter, M. G., Parker, A., Lindner, C. C., Izquierdo, A. D., & Murray, E. A. (2000). Control of response selection by reinforcer value requires interaction of amygdala and orbital prefrontal cortex. J Neurosci, 20(11), 4311‐4319. Bechara, A., Damasio, H., Damasio, A. R., & Lee, G. P. (1999). Different contributions of the human amygdala and ventromedial prefrontal cortex to decision‐making. J Neurosci, 19(13), 5473‐5481. Beckmann, M., Johansen‐Berg, H., & Rushworth, M. F. (2009). Connectivity‐based parcellation of human cingulate cortex and its relation to functional specialization. J Neurosci, 29(4), 1175‐1190. 310 Behan, M., & Haberly, L. B. (1999). Intrinsic and efferent connections of the endopiriform nucleus in rat. J Comp Neurol, 408(4), 532‐548. Benchenane, K., Peyrache, A., Khamassi, M., Tierney, P. L., Gioanni, Y., Battaglia, F. P., et al. (2010). Coherent theta oscillations and reorganization of spike timing in the hippocampal‐ prefrontal network upon learning. Neuron, 66(6), 921‐936. Bentivoglio, M., Kultas‐Ilinsky, K., & Ilinsky, I. (1993). Limbic thalamus: structure, intrinsic organization, and connections. In B. A. Vogt & M. Gabriel (Eds.), Neurobiology of cingulate cortex and limbic thalamus (pp. 71‐122). Boston: Birkhauser. Berens, P., Keliris, G. A., Ecker, A. S., Logothetis, N. K., & Tolias, A. S. (2008). Comparing the feature selectivity of the gamma‐band of the local field potential and the underlying spiking activity in primate visual cortex. Front Syst Neurosci, 2, 2. Berens, P., Keliris, G. A., Ecker, A. S., Logothetis, N. K., & Tolias, A. S. (2008). Feature selectivity of the gamma‐band of the local field potential in primate primary visual cortex. Front Neurosci, 2(2), 199‐207. Bergfield, K. L., Hanson, K. D., Chen, K., Teipel, S. J., Hampel, H., Rapoport, S. I., et al. (2010). Age‐related networks of regional covariance in MRI gray matter: reproducible multivariate patterns in healthy aging. Neuroimage, 49(2), 1750‐ 1759. Berlanga, M. L., Simpson, T. K., & Alcantara, A. A. (2005). Dopamine D5 receptor localization on cholinergic neurons of the rat forebrain and diencephalon: a potential neuroanatomical substrate involved in mediating dopaminergic influences on acetylcholine release. J Comp Neurol, 492(1), 34‐49. Beversdorf, D. Q., Hughes, J. D., Steinberg, B. A., Lewis, L. D., & Heilman, K. M. (1999). Noradrenergic modulation of cognitive flexibility in problem solving. Neuroreport, 10(13), 2763‐2767. Birrell, J. M., & Brown, V. J. (2000). Medial frontal cortex mediates perceptual attentional set shifting in the rat. J Neurosci, 20(11), 4320‐4324. Borgers, C., Epstein, S., & Kopell, N. J. (2005). Background gamma rhythmicity and attention in cortical local circuits: a computational study. Proc Natl Acad Sci U S A, 102(19), 7002‐7007. 311 Borgers, C., & Kopell, N. (2005). Effects of noisy drive on rhythms in networks of excitatory and inhibitory neurons. Neural Comput, 17(3), 557‐608. Borges, J. L. (1949/1999). The doctorine of cycles. In E. Weinberger (Ed.), Jorge Luis Borges Selected Non‐Fictions (pp. 115‐122). New York: Penguin Putnam. Bostan, A. C., Dum, R. P., & Strick, P. L. (2010). The basal ganglia communicate with the cerebellum. Proc Natl Acad Sci U S A, 107(18), 8452‐8456. Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Cohen, J. D. (2001). Conflict monitoring and cognitive control. Psychol Rev, 108(3), 624‐652. Botvinick, M. M., Cohen, J. D., & Carter, C. S. (2004). Conflict monitoring and anterior cingulate cortex: an update. Trends Cogn Sci, 8(12), 539‐546. Boulougouris, V., Dalley, J. W., & Robbins, T. W. (2007). Effects of orbitofrontal, infralimbic and prelimbic cortical lesions on serial spatial reversal learning in the rat. Behav Brain Res, 179(2), 219‐228. Bouthenet, M. L., Souil, E., Martres, M. P., Sokoloff, P., Giros, B., & Schwartz, J. C. (1991). Localization of dopamine D3 receptor mRNA in the rat brain using in situ hybridization histochemistry: comparison with dopamine D2 receptor mRNA. Brain Res, 564(2), 203‐219. Bragin, A., Jando, G., Nadasdy, Z., Hetke, J., Wise, K., & Buzsaki, G. (1995). Gamma (40‐ 100 Hz) oscillation in the hippocampus of the behaving rat. J Neurosci, 15(1 Pt 1), 47‐60. Braver, T. S., & Cohen, J. D. (2000). On the control of control: the role of dopamine in regulating prefrontal function and working memory. In S. Monsell & J. Driver (Eds.), Control of Cognitive Processes. Cambridge: MIT Press. Bressler, S. L., & Freeman, W. J. (1980). Frequency analysis of olfactory system EEG in cat, rabbit, and rat. Electroencephalogr Clin Neurophysiol, 50(1‐2), 19‐24. Brock, L. G., Coombs, J. S., & Eccles, J. C. (1952). The nature of the monosynaptic excitatory and inhibitory processes in the spinal cord. Proc R Soc Lond B Biol Sci, 140(899), 170‐176. Brock, L. G., Coombs, J. S., & Eccles, J. C. (1952). Synaptic excitation and inhibition. J Physiol, 117(2), 8p. 312 Brodmann, K. (1909). Vergleichende Lokalisationslehre der Grosshirnrinde in ihren Prinzipien dargestellt auf Grund des Zellenbaues (L. J. Garey, Trans. 3 ed.). New York: Springer Science and Business Media. Brown, C. H. (1984). Directional hearing in aging rats. Exp Aging Res, 10(1), 35‐38. Brunel, N., & Wang, X. J. (2003). What determines the frequency of fast network oscillations with irregular neural discharges? I. Synaptic dynamics and excitation‐ inhibition balance. J Neurophysiol, 90(1), 415‐430. Brushfield, A. M., Luu, T. T., Callahan, B. D., & Gilbert, P. E. (2008). A comparison of discrimination and reversal learning for olfactory and visual stimuli in aged rats. Behav Neurosci, 122(1), 54‐62. Buckley, M. J., Mansouri, F. A., Hoda, H., Mahboubi, M., Browning, P. G., Kwok, S. C., et al. (2009). Dissociable components of rule‐guided behavior depend on distinct medial and prefrontal regions. Science, 325(5936), 52‐58. Buckley, M. J., Mansouri, F. A., Hoda, H., Mahboubi, M., Browning, P. G., Kwok, S. C., et al. (2009). Dissociable components of rule‐guided behavior depend on distinct medial and prefrontal regions. Science, 325(5936), 52‐58. Buhot, M. C. (1997). Serotonin receptors in cognitive behaviors. Curr Opin Neurobiol, 7(2), 243‐254. Bush, G., Luu, P., & Posner, M. I. (2000). Cognitive and emotional influences in anterior cingulate cortex. Trends Cogn Sci, 4(6), 215‐222. Buzsaki, G. (2006). Rhythms in the Brain. New York: Oxford University Press. Cabeza, R., Anderson, N. D., Locantore, J. K., & McIntosh, A. R. (2002). Aging gracefully: compensatory brain activity in high‐performing older adults. Neuroimage, 17(3), 1394‐1402. Cabeza, R., McIntosh, A. R., Tulving, E., Nyberg, L., & Grady, C. L. (1997). Age‐related differences in effective neural connectivity during encoding and recall. Neuroreport, 8(16), 3479‐3483. 313 Cardin, J. A., Carlen, M., Meletis, K., Knoblich, U., Zhang, F., Deisseroth, K., et al. (2009). Driving fast‐spiking cells induces gamma rhythm and controls sensory responses. Nature, 459(7247), 663‐667. Carr, D. B., & Sesack, S. R. (2000). Dopamine terminals synapse on callosal projection neurons in the rat prefrontal cortex. J Comp Neurol, 425(2), 275‐283. Carr, D. B., & Sesack, S. R. (2000). GABA‐containing neurons in the rat ventral tegmental area project to the prefrontal cortex. Synapse, 38(2), 114‐123. Carr, D. B., & Sesack, S. R. (2000). Projections from the rat prefrontal cortex to the ventral tegmental area: target specificity in the synaptic associations with mesoaccumbens and mesocortical neurons. J Neurosci, 20(10), 3864‐3873. Casu, M. A., Wong, T. P., De Koninck, Y., Ribeiro‐da‐Silva, A., & Cuello, A. C. (2002). Aging causes a preferential loss of cholinergic innervation of characterized neocortical pyramidal neurons. Cereb Cortex, 12(3), 329‐337. Chahal, H. S., & Drake, W. M. (2007). The endocrine system and ageing. J Pathol, 211(2), 173‐180. Chang, Y. M., Rosene, D. L., Killiany, R. J., Mangiamele, L. A., & Luebke, J. I. (2005). Increased action potential firing rates of layer 2/3 pyramidal cells in the prefrontal cortex are significantly related to cognitive performance in aged monkeys. Cereb Cortex, 15(4), 409‐418. Chrobak, J. J., & Buzsaki, G. (1998). Gamma oscillations in the entorhinal cortex of the freely behaving rat. J Neurosci, 18(1), 388‐398. Chua, E. F., Schacter, D. L., Rand‐Giovannetti, E., & Sperling, R. A. (2006). Understanding metamemory: neural correlates of the cognitive process and subjective level of confidence in recognition memory. Neuroimage, 29(4), 1150‐1160. Chudasama, Y., Passetti, F., Rhodes, S. E., Lopian, D., Desai, A., & Robbins, T. W. (2003). Dissociable aspects of performance on the 5‐choice serial reaction time task following lesions of the dorsal anterior cingulate, infralimbic and orbitofrontal cortex in the rat: differential effects on selectivity, impulsivity and compulsivity. Behav Brain Res, 146(1‐2), 105‐119. Chudasama, Y., & Robbins, T. W. (2003). Dissociable contributions of the orbitofrontal and infralimbic cortex to pavlovian autoshaping and discrimination reversal 314 learning: further evidence for the functional heterogeneity of the rodent frontal cortex. J Neurosci, 23(25), 8771‐8780. Cohen, J. D., Braver, T. S., & O'Reilly, R. C. (1996). A computational approach to prefrontal cortex, cognitive control and schizophrenia: recent developments and current challenges. Philos Trans R Soc Lond B Biol Sci, 351(1346), 1515‐1527. Colgin, L. L., Denninger, T., Fyhn, M., Hafting, T., Bonnevie, T., Jensen, O., et al. (2009). Frequency of gamma oscillations routes flow of information in the hippocampus. Nature, 462(7271), 353‐357. Conde, F., Maire‐Lepoivre, E., Audinat, E., & Crepel, F. (1995). Afferent connections of the medial frontal cortex of the rat. II. Cortical and subcortical afferents. J Comp Neurol, 352(4), 567‐593. Corbit, L. H., & Balleine, B. W. (2003). The role of prelimbic cortex in instrumental conditioning. Behav Brain Res, 146(1‐2), 145‐157. Cowen, S. L., & McNaughton, B. L. (2007). Selective delay activity in the medial prefrontal cortex of the rat: contribution of sensorimotor information and contingency. J Neurophysiol, 98(1), 303‐316. Craver, C. F. (2002). Interlevel experiments and multilevel mechanisms in the neuroscience of memory. Philosophy of Science, 69, 583‐597. Craver, C. F. (2005). Beyond reduction: mechanisms, multifield integration and the unity of neuroscience. Studies in History and Philosophy of Biological and Biomedical Sciences, 36, 373‐395. Crick, F. C., & Koch, C. (2005). What is the function of the claustrum? Philos Trans R Soc Lond B Biol Sci, 360(1458), 1271‐1279. Csicsvari, J., Jamieson, B., Wise, K. D., & Buzsaki, G. (2003). Mechanisms of gamma oscillations in the hippocampus of the behaving rat. Neuron, 37(2), 311‐322. Dalley, J. W., Cardinal, R. N., & Robbins, T. W. (2004). Prefrontal executive and cognitive functions in rodents: neural and neurochemical substrates. Neurosci Biobehav Rev, 28(7), 771‐784. Damasio, A. R. (1994). Decartes' Error. New York: Avon Books. 315 Davis, S. W., Dennis, N. A., Daselaar, S. M., Fleck, M. S., & Cabeza, R. (2008). Que PASA? The posterior‐anterior shift in aging. Cereb Cortex, 18(5), 1201‐1209. Daw, N. D., Niv, Y., & Dayan, P. (2005). Uncertainty‐based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci, 8(12), 1704‐1711. Dayan, P., & Balleine, B. W. (2002). Reward, motivation, and reinforcement learning. Neuron, 36(2), 285‐298. de Wit, S., Kosaki, Y., Balleine, B. W., & Dickinson, A. (2006). Dorsomedial prefrontal cortex resolves response conflict in rats. J Neurosci, 26(19), 5224‐5229. Deans, M. R., Gibson, J. R., Sellitto, C., Connors, B. W., & Paul, D. L. (2001). Synchronous activity of inhibitory networks in neocortex requires electrical synapses containing connexin36. Neuron, 31(3), 477‐485. DeCoteau, W. E., Kesner, R. P., & Williams, J. M. (1997). Short‐term memory for food reward magnitude: the role of the prefrontal cortex. Behav Brain Res, 88(2), 239‐ 249. Degos, B., Deniau, J. M., Le Cam, J., Mailly, P., & Maurice, N. (2008). Evidence for a direct subthalamo‐cortical loop circuit in the rat. Eur J Neurosci, 27(10), 2599‐2610. Delatour, B., & Gisquet‐Verrier, P. (2001). Involvement of the dorsal anterior cingulate cortex in temporal behavioral sequencing: subregional analysis of the medial prefrontal cortex in rat. Behav Brain Res, 126(1‐2), 105‐114. Dias, R., & Aggleton, J. P. (2000). Effects of selective excitotoxic prefrontal lesions on acquisition of nonmatching‐ and matching‐to‐place in the T‐maze in the rat: differential involvement of the prelimbic‐infralimbic and anterior cingulate cortices in providing behavioural flexibility. Eur J Neurosci, 12(12), 4457‐4466. Donoghue, J. P., & Wise, S. P. (1982). The motor cortex of the rat: cytoarchitecture and microstimulation mapping. J Comp Neurol, 212(1), 76‐88. Draguhn, A., Traub, R. D., Schmitz, D., & Jefferys, J. G. (1998). Electrical coupling underlies high‐frequency oscillations in the hippocampus in vitro. Nature, 394(6689), 189‐192. 316 Dulaney, C. L., & Rogers, W. A. (1994). Mechanisms underlying reduction in Stroop interference with practice for young and old adults. J Exp Psychol Learn Mem Cogn, 20(2), 470‐484. Durstewitz, D., & Seamans, J. K. (2002). The computational role of dopamine D1 receptors in working memory. Neural Netw, 15(4‐6), 561‐572. Durstewitz, D., Seamans, J. K., & Sejnowski, T. J. (2000). Dopamine‐mediated stabilization of delay‐period activity in a network model of prefrontal cortex. J Neurophysiol, 83(3), 1733‐1750. Durstewitz, D., Seamans, J. K., & Sejnowski, T. J. (2000). Neurocomputational models of working memory. Nat Neurosci, 3 Suppl, 1184‐1191. Durstewitz, D., Vittoz, N. M., Floresco, S. B., & Seamans, J. K. (2010). Abrupt transitions between prefrontal neural ensemble states accompany behavioral transitions during rule learning. Neuron, 66(3), 438‐448. Ecker, A. S., Berens, P., Keliris, G. A., Bethge, M., Logothetis, N. K., & Tolias, A. S. (2010). Decorrelated neuronal firing in cortical microcircuits. Science, 327(5965), 584‐ 587. Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M., et al. (1988). Coherent oscillations: a mechanism of feature linking in the visual cortex? Multiple electrode and correlation analyses in the cat. Biol Cybern, 60(2), 121‐ 130. Edelman, G. M. (1993). Neural Darwinism: selection and reentrant signaling in higher brain function. Neuron, 10(2), 115‐125. Eeckman, F. H., & Freeman, W. J. (1990). Correlations between unit firing and EEG in the rat olfactory system. Brain Res, 528(2), 238‐244. Engel, A. K., Konig, P., Kreiter, A. K., & Singer, W. (1991). Interhemispheric synchronization of oscillatory neuronal responses in cat visual cortex. Science, 252(5010), 1177‐1179. Engel, A. K., Kreiter, A. K., Konig, P., & Singer, W. (1991). Synchronization of oscillatory neuronal responses between striate and extrastriate visual cortical areas of the cat. Proc Natl Acad Sci U S A, 88(14), 6048‐6052. 317 Erickson, C. A., Jung, M. W., McNaughton, B. L., & Barnes, C. A. (1996). Contribution of single‐unit spike waveform changes to temperature‐induced alterations in hippocampal population spikes. Exp Brain Res, 107(3), 348‐360. Euston, D. R., & McNaughton, B. L. (2006). Apparent encoding of sequential context in rat medial prefrontal cortex is accounted for by behavioral variability. J Neurosci, 26(51), 13143‐13155. Falkenstein, M., Hoormann, J., Christ, S., & Hohnsbein, J. (2000). ERP components on reaction errors and their functional significance: a tutorial. Biol Psychol, 51(2‐3), 87‐107. Falkenstein, M., Hoormann, J., & Hohnsbein, J. (2001). Changes of error‐related ERPs with age. Exp Brain Res, 138(2), 258‐262. Fanselow, M. S., & Dong, H. W. (2010). Are the dorsal and ventral hippocampus functionally distinct structures? Neuron, 65(1), 7‐19. Felleman, D. J., & Van Essen, D. C. (1991). Distributed hierarchical processing in the primate cerebral cortex. Cereb Cortex, 1(1), 1‐47. Fellows, L. K., & Farah, M. J. (2005). Is anterior cingulate cortex necessary for cognitive control? Brain, 128(Pt 4), 788‐796. Fisk, G. D., & Wyss, J. M. (1999). Associational projections of the anterior midline cortex in the rat: intracingulate and retrosplenial connections. Brain Res, 825(1‐2), 1‐13. Floresco, S. B., & Tse, M. T. (2007). Dopaminergic regulation of inhibitory and excitatory transmission in the basolateral amygdala‐prefrontal cortical pathway. J Neurosci, 27(8), 2045‐2057. Foote, S. L., Bloom, F. E., & Aston‐Jones, G. (1983). Nucleus locus ceruleus: new evidence of anatomical and physiological specificity. Physiol Rev, 63(3), 844‐914. Frank, M. J., & Claus, E. D. (2006). Anatomy of a decision: striato‐orbitofrontal interactions in reinforcement learning, decision making, and reversal. Psychol Rev, 113(2), 300‐326. Frank, M. J., Moustafa, A. A., Haughey, H. M., Curran, T., & Hutchison, K. E. (2007). Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc Natl Acad Sci U S A, 104(41), 16311‐16316. 318 Frankland, P. W., Bontempi, B., Talton, L. E., Kaczmarek, L., & Silva, A. J. (2004). The involvement of the anterior cingulate cortex in remote contextual fear memory. Science, 304(5672), 881‐883. Freedman, L. J., Insel, T. R., & Smith, Y. (2000). Subcortical projections of area 25 (subgenual cortex) of the macaque monkey. J Comp Neurol, 421(2), 172‐188. Freund, T. F., & Gulyas, A. I. (1991). GABAergic interneurons containing calbindin D28K or somatostatin are major targets of GABAergic basal forebrain afferents in the rat neocortex. J Comp Neurol, 314(1), 187‐199. Frick, K. M. (2009). Estrogens and age‐related memory decline in rodents: what have we learned and where do we go from here? Horm Behav, 55(1), 2‐23. Fries, P. (2005). A mechanism for cognitive dynamics: neuronal communication through neuronal coherence. Trends Cogn Sci, 9(10), 474‐480. Fries, P. (2009). Neuronal gamma‐band synchronization as a fundamental process in cortical computation. Annu Rev Neurosci, 32, 209‐224. Fritts, M. E., Asbury, E. T., Horton, J. E., & Isaac, W. L. (1998). Medial prefrontal lesion deficits involving or sparing the prelimbic area in the rat. Physiol Behav, 64(3), 373‐380. Fuster, J. M. (1973). Unit activity in prefrontal cortex during delayed‐response performance: neuronal correlates of transient memory. J Neurophysiol, 36(1), 61‐78. Fuster, J. M. (1980). The Prefrontal Cortex: Anatomy, Physiology, and Neuropsychology of the Frontal Lobe. New York: Raven Press. Fuster, J. M. (2008). The Prefrontal Cortex (4 ed.). New York: Academic Press. Fuster, J. M., & Alexander, G. E. (1971). Neuron activity related to short‐term memory. Science, 173(997), 652‐654. Fuster, J. M., Bauer, R. H., & Jervey, J. P. (1985). Functional interactions between inferotemporal and prefrontal cortex in a cognitive task. Brain Res, 330(2), 299‐ 307. 319 Gabbott, P. L., Dickie, B. G., Vaid, R. R., Headlam, A. J., & Bacon, S. J. (1997). Local‐circuit neurones in the medial prefrontal cortex (areas 25, 32 and 24b) in the rat: morphology and quantitative distribution. J Comp Neurol, 377(4), 465‐499. Gabbott, P. L., Warner, T. A., Jays, P. R., Salway, P., & Busby, S. J. (2005). Prefrontal cortex in the rat: projections to subcortical autonomic, motor, and limbic centers. J Comp Neurol, 492(2), 145‐177. Gaffan, D. (2002). Against memory systems. Philos Trans R Soc Lond B Biol Sci, 357(1424), 1111‐1121. Gaspar, P., Bloch, B., & Le Moine, C. (1995). D1 and D2 receptor gene expression in the rat frontal cortex: cellular localization in different classes of efferent neurons. Eur J Neurosci, 7(5), 1050‐1063. Gaykema, R. P., Luiten, P. G., Nyakas, C., & Traber, J. (1990). Cortical projection patterns of the medial septum‐diagonal band complex. J Comp Neurol, 293(1), 103‐124. Gaykema, R. P., van Weeghel, R., Hersh, L. B., & Luiten, P. G. (1991). Prefrontal cortical projections to the cholinergic neurons in the basal forebrain. J Comp Neurol, 303(4), 563‐583. Gazzaley, A., & D'Esposito, M. (2007). Top‐down modulation and normal aging. Ann N Y Acad Sci, 1097, 67‐83. Gazzaley, A., Rissman, J., Cooney, J., Rutman, A., Seibert, T., Clapp, W., et al. (2007). Functional interactions between prefrontal and visual association cortex contribute to top‐down modulation of visual processing. Cereb Cortex, 17 Suppl 1, i125‐135. Gerfen, C. R. (2004). Basal Ganglia. In G. Paxinos (Ed.), The Rat Nervous System (pp. 458‐ 509). San Diego: Elsevier Academic Press. Gibson, J. R., Beierlein, M., & Connors, B. W. (1999). Two networks of electrically coupled inhibitory neurons in neocortex. Nature, 402(6757), 75‐79. Giguere, M., & Goldman‐Rakic, P. S. (1988). Mediodorsal nucleus: areal, laminar, and tangential distribution of afferents and efferents in the frontal lobe of rhesus monkeys. J Comp Neurol, 277(2), 195‐213. 320 Gioanni, Y., Rougeot, C., Clarke, P. B., Lepouse, C., Thierry, A. M., & Vidal, C. (1999). Nicotinic receptors in the rat prefrontal cortex: increase in glutamate release and facilitation of mediodorsal thalamo‐cortical transmission. Eur J Neurosci, 11(1), 18‐30. Gioanni, Y., Rougeot, C., Clarke, P. B., Lepouse, C., Thierry, A. M., & Vidal, C. (1999). Nicotinic receptors in the rat prefrontal cortex: increase in glutamate release and facilitation of mediodorsal thalamo‐cortical transmission. Eur J Neurosci, 11(1), 18‐30. Gisquet‐Verrier, P., & Delatour, B. (2006). The role of the rat prelimbic/infralimbic cortex in working memory: not involved in the short‐term maintenance but in monitoring and processing functions. Neuroscience, 141(2), 585‐596. Gleich, O., Weiss, M., & Strutz, J. (2004). Age‐dependent changes in the lateral superior olive of the gerbil (Meriones unguiculatus). Hear Res, 194(1‐2), 47‐59. Goldberg, G. (1985). Supplementary motor area structure and function: review and hypotheses. Behavioral and Brain Sciences, 8, 567‐616. Goldman‐Rakic, P. S., & Brown, R. M. (1981). Regional changes of monoamines in cerebral cortex and subcortical structures of aging rhesus monkeys. Neuroscience, 6(2), 177‐187. Grady, C. L., & Craik, F. I. (2000). Changes in memory processing with age. Curr Opin Neurobiol, 10(2), 224‐231. Gray, C. M., Konig, P., Engel, A. K., & Singer, W. (1989). Oscillatory responses in cat visual cortex exhibit inter‐columnar synchronization which reflects global stimulus properties. Nature, 338(6213), 334‐337. Gray, C. M., & Singer, W. (1989). Stimulus‐specific neuronal oscillations in orientation columns of cat visual cortex. Proc Natl Acad Sci U S A, 86(5), 1698‐1702. Grieve, S. M., Clark, C. R., Williams, L. M., Peduto, A. J., & Gordon, E. (2005). Preservation of limbic and paralimbic structures in aging. Hum Brain Mapp, 25(4), 391‐401. Grill, J. D., & Riddle, D. R. (2002). Age‐related and laminar‐specific dendritic changes in the medial frontal cortex of the rat. Brain Res, 937(1‐2), 8‐21. 321 Grothe, B., Pecka, M., & McAlpine, D. (2010). Mechanisms of sound localization in mammals. Physiol Rev, 90(3), 983‐1012. Haddon, J. E., & Killcross, A. S. (2005). Medial prefrontal cortex lesions abolish contextual control of competing responses. J Exp Anal Behav, 84(3), 485‐504. Hadland, K. A., Rushworth, M. F., Gaffan, D., & Passingham, R. E. (2003). The anterior cingulate and reward‐guided selection of actions. J Neurophysiol, 89(2), 1161‐ 1164. Hartline, H. K., & Ratliff, F. (1954). Spatial summation of inhibitory influences in the eye of limulus. Science, 120(3124), 781. Hartline, H. K., & Ratliff, F. (1957). Inhibitory interaction of receptor units in the eye of Limulus. J Gen Physiol, 40(3), 357‐376. Hartline, H. K., Wagner, H. G., & Ratliff, F. (1956). Inhibition in the eye of Limulus. J Gen Physiol, 39(5), 651‐673. Hasher, L., & Zacks, R. T. (1988). Working memory, comprehension, and aging: a review and a new view. In G. H. Bower (Ed.), The Psychology of Learning and Motivation (Vol. 22, pp. 193‐225). New York: Academic Press. Hasselmo, M. E., Bodelon, C., & Wyble, B. P. (2002). A proposed function for hippocampal theta rhythm: separate phases of encoding and retrieval enhance reversal of prior learning. Neural Comput, 14(4), 793‐817. Hasselmo, M. E., & Bower, J. M. (1993). Acetylcholine and memory. Trends Neurosci, 16(6), 218‐222. Hasselmo, M. E., Hay, J., Ilyn, M., & Gorchetchnikov, A. (2002). Neuromodulation, theta rhythm and rat spatial navigation. Neural Netw, 15(4‐6), 689‐707. Hawkins, J., & Blakeslee, S. (2004). On Intelligence. New York: Henry Hold and Company, LLC. Hayden, B. Y., Pearson, J. M., & Platt, M. L. (2009). Fictive reward signals in the anterior cingulate cortex. Science, 324(5929), 948‐950. Hazy, T. E., Frank, M. J., & O'Reilly, R. C. (2006). Banishing the homunculus: making working memory work. Neuroscience, 139(1), 105‐118. 322 Hebb, D. O. (1949). The Organization of Behavior: A Neuropsychological Theory. Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Heidbreder, C. A., & Groenewegen, H. J. (2003). The medial prefrontal cortex in the rat: evidence for a dorso‐ventral distinction based upon functional and anatomical characteristics. Neurosci Biobehav Rev, 27(6), 555‐579. Henke, P. G. (1990). Hippocampal pathway to the amygdala and stress ulcer development. Brain Res Bull, 25(5), 691‐695. Henny, P., & Jones, B. E. (2008). Projections from basal forebrain to prefrontal cortex comprise cholinergic, GABAergic and glutamatergic inputs to pyramidal cells or interneurons. Eur J Neurosci, 27(3), 654‐670. Henze, D. A., Gonzalez‐Burgos, G. R., Urban, N. N., Lewis, D. A., & Barrionuevo, G. (2000). Dopamine increases excitability of pyramidal neurons in primate prefrontal cortex. J Neurophysiol, 84(6), 2799‐2809. Herman, J. P., & Cullinan, W. E. (1997). Neurocircuitry of stress: central control of the hypothalamo‐pituitary‐adrenocortical axis. Trends Neurosci, 20(2), 78‐84. Herzog, C. D., Nowak, K. A., Sarter, M., & Bruno, J. P. (2003). Microdialysis without acetylcholinesterase inhibition reveals an age‐related attenuation in stimulated cortical acetylcholine release. Neurobiol Aging, 24(6), 861‐863. Hillman, K. L., & Bilkey, D. K. (2010). Neurons in the rat anterior cingulate cortex dynamically encode cost‐benefit in a spatial decision‐making task. J Neurosci, 30(22), 7705‐7713. Hok, V., Save, E., Lenck‐Santini, P. P., & Poucet, B. (2005). Coding for spatial goals in the prelimbic/infralimbic area of the rat frontal cortex. Proc Natl Acad Sci U S A, 102(12), 4602‐4607. Holroyd, C. B., & Coles, M. G. (2002). The neural basis of human error processing: reinforcement learning, dopamine, and the error‐related negativity. Psychol Rev, 109(4), 679‐709. Hoover, W. B., & Vertes, R. P. (2007). Anatomical analysis of afferent projections to the medial prefrontal cortex in the rat. Brain Struct Funct, 212(2), 149‐179. 323 Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci U S A, 79(8), 2554‐2558. Hyman, J. M., Zilli, E. A., Paley, A. M., & Hasselmo, M. E. (2005). Medial prefrontal cortex cells show dynamic modulation with the hippocampal theta rhythm dependent on behavior. Hippocampus, 15(6), 739‐749. Inestrosa, N. C., Reyes, A. E., Chacon, M. A., Cerpa, W., Villalon, A., Montiel, J., et al. (2005). Human‐like rodent amyloid‐beta‐peptide determines Alzheimer pathology in aged wild‐type Octodon degu. Neurobiol Aging, 26(7), 1023‐1028. Ingham, N. J., Thornton, S. K., & Withington, D. J. (1998). Age‐related changes in auditory spatial properties of the guinea pig superior colliculus. Brain Res, 788(1‐ 2), 60‐68. Innocenti, G. M., & Vercelli, A. (2010). Dendritic bundles, minicolumns, columns, and cortical output units. Front Neuroanat, 4, 11. Insausti, R., Herrero, M. T., & Witter, M. P. (1997). Entorhinal cortex of the rat: cytoarchitectonic subdivisions and the origin and distribution of cortical efferents. Hippocampus, 7(2), 146‐183. Jacobson, L., & Sapolsky, R. (1991). The role of the hippocampus in feedback regulation of the hypothalamic‐pituitary‐adrenocortical axis. Endocr Rev, 12(2), 118‐134. Jankowski, M. P., & Sesack, S. R. (2004). Prefrontal cortical projections to the rat dorsal raphe nucleus: ultrastructural features and associations with serotonin and gamma‐aminobutyric acid neurons. J Comp Neurol, 468(4), 518‐529. Jenkins, I. H., Brooks, D. J., Nixon, P. D., Frackowiak, R. S., & Passingham, R. E. (1994). Motor sequence learning: a study with positron emission tomography. J Neurosci, 14(6), 3775‐3790. Jimura, K., & Braver, T. S. (2010). Age‐related shifts in brain activity dynamics during task switching. Cereb Cortex, 20(6), 1420‐1431. Johnson, A., & Redish, A. D. (2007). Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J Neurosci, 27(45), 12176‐12189. 324 Jones, B. F., Groenewegen, H. J., & Witter, M. P. (2005). Intrinsic connections of the cingulate cortex in the rat suggest the existence of multiple functionally segregated networks. Neuroscience, 133(1), 193‐207. Jones, B. F., & Witter, M. P. (2007). Cingulate cortex projections to the parahippocampal region and hippocampal formation in the rat. Hippocampus, 17(10), 957‐976. Jones, M. W., & Wilson, M. A. (2005). Phase precession of medial prefrontal cortical activity relative to the hippocampal theta rhythm. Hippocampus, 15(7), 867‐873. Jones, M. W., & Wilson, M. A. (2005). Theta rhythms coordinate hippocampal‐prefrontal interactions in a spatial memory task. PLoS Biol, 3(12), e402. Jung, M. W., Qin, Y., McNaughton, B. L., & Barnes, C. A. (1998). Firing characteristics of deep layer neurons in prefrontal cortex in rats performing spatial working memory tasks. Cereb Cortex, 8(5), 437‐450. Kaas, J. H., & Hackett, T. A. (2000). Subdivisions of auditory cortex and processing streams in primates. Proc Natl Acad Sci U S A, 97(22), 11793‐11799. Kennerley, S. W., Dahmubed, A. F., Lara, A. H., & Wallis, J. D. (2009). Neurons in the frontal lobe encode the value of multiple decision variables. J Cogn Neurosci, 21(6), 1162‐1178. Kennerley, S. W., & Wallis, J. D. (2009). Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables. Eur J Neurosci, 29(10), 2061‐2073. Kennerley, S. W., Walton, M. E., Behrens, T. E., Buckley, M. J., & Rushworth, M. F. (2006). Optimal decision making and the anterior cingulate cortex. Nat Neurosci, 9(7), 940‐947. Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008). Neural correlates, computation and behavioural impact of decision confidence. Nature, 455(7210), 227‐231. Kesner, R. P., Hunt, M. E., Williams, J. M., & Long, J. M. (1996). Prefrontal cortex and working memory for spatial response, spatial location, and visual object information in the rat. Cereb Cortex, 6(2), 311‐318. 325 Killcross, S., & Coutureau, E. (2003). Coordination of actions and habits in the medial prefrontal cortex of rats. Cereb Cortex, 13(4), 400‐408. Kim, H. G., & Connors, B. W. (1993). Apical dendrites of the neocortex: correlation between sodium‐ and calcium‐dependent spiking and pyramidal cell morphology. J Neurosci, 13(12), 5301‐5311. Kim, J. J., Clark, R. E., & Thompson, R. F. (1995). Hippocampectomy impairs the memory of recently, but not remotely, acquired trace eyeblink conditioned responses. Behav Neurosci, 109(2), 195‐203. Kim, J. J., & Fanselow, M. S. (1992). Modality‐specific retrograde amnesia of fear. Science, 256(5057), 675‐677. Koch, C., Rapp, M., & Segev, I. (1996). A brief history of time (constants). Cereb Cortex, 6(2), 93‐101. Kopell, N., & LeMasson, G. (1994). Rhythmogenesis, amplitude modulation, and multiplexing in a cortical architecture. Proc Natl Acad Sci U S A, 91(22), 10586‐ 10590. Kramer, A. F., Humphrey, D. G., Larish, J. F., Logan, G. D., & Strayer, D. L. (1994). Aging and inhibition: beyond a unitary view of inhibitory processing in attention. Psychol Aging, 9(4), 491‐512. Kramer, A. F., & Kray, J. (2006). Aging and attention. In E. Bialystok & F. I. M. Craik (Eds.), Lifespan Cognition, Mechanisms of Change (pp. 57‐69). 2006: Oxford University Press. Krettek, J. E., & Price, J. L. (1977). The cortical projections of the mediodorsal nucleus and adjacent thalamic nuclei in the rat. J Comp Neurol, 171(2), 157‐191. Kuroda, M., Yokofujita, J., & Murakami, K. (1998). An ultrastructural study of the neural circuit between the prefrontal cortex and the mediodorsal nucleus of the thalamus. Prog Neurobiol, 54(4), 417‐458. Kyd, R. J., & Bilkey, D. K. (2003). Prefrontal cortex lesions modify the spatial properties of hippocampal place cells. Cereb Cortex, 13(5), 444‐451. 326 Kyd, R. J., & Bilkey, D. K. (2005). Hippocampal place cells show increased sensitivity to changes in the local environment following prefrontal cortex lesions. Cereb Cortex, 15(6), 720‐731. Lambe, E. K., Picciotto, M. R., & Aghajanian, G. K. (2003). Nicotine induces glutamate release from thalamocortical terminals in prefrontal cortex. Neuropsychopharmacology, 28(2), 216‐225. Langenecker, S. A., Nielson, K. A., & Rao, S. M. (2004). fMRI of healthy older adults during Stroop interference. Neuroimage, 21(1), 192‐200. Larson, E. R., & Ariano, M. A. (1995). D3 and D2 dopamine receptors: visualization of cellular expression patterns in motor and limbic structures. Synapse, 20(4), 325‐ 337. Lavin, A., Nogueira, L., Lapish, C. C., Wightman, R. M., Phillips, P. E., & Seamans, J. K. (2005). Mesocortical dopamine neurons operate in distinct temporal domains using multimodal signaling. J Neurosci, 25(20), 5013‐5023. Lei, W., Jiao, Y., Del Mar, N., & Reiner, A. (2004). Evidence for differential cortical input to direct pathway versus indirect pathway striatal projection neurons in rats. J Neurosci, 24(38), 8289‐8299. Lena, I., Parrot, S., Deschaux, O., Muffat‐Joly, S., Sauvinet, V., Renaud, B., et al. (2005). Variations in extracellular levels of dopamine, noradrenaline, glutamate, and aspartate across the sleep‐‐wake cycle in the medial prefrontal cortex and nucleus accumbens of freely moving rats. J Neurosci Res, 81(6), 891‐899. Lidow, M. S., Wang, F., Cao, Y., & Goldman‐Rakic, P. S. (1998). Layer V neurons bear the majority of mRNAs encoding the five distinct dopamine receptor subtypes in the primate prefrontal cortex. Synapse, 28(1), 10‐20. Lie, C. H., Specht, K., Marshall, J. C., & Fink, G. R. (2006). Using fMRI to decompose the neural processes underlying the Wisconsin Card Sorting Test. Neuroimage, 30(3), 1038‐1049. Liotti, M., Woldorff, M. G., Perez, R., & Mayberg, H. S. (2000). An ERP study of the temporal course of the Stroop color‐word interference effect. Neuropsychologia, 38(5), 701‐711. 327 Lisman, J. E., Fellous, J. M., & Wang, X. J. (1998). A role for NMDA‐receptor channels in working memory. Nat Neurosci, 1(4), 273‐275. Logan, C. A. (1999). The altered rationale for the choice of a standard animal in experimental psychology: Henry H. Donaldson, Adolf Meyer, and "the" albino rat. Hist Psychol, 2(1), 3‐24. Logan, C. A. (2005). The legacy of Adolf Meyer's comparative approach: Worcester rats and the strange birth of the animal model. Integr Physiol Behav Sci, 40(4), 169‐ 181. Logan, J. M., Sanders, A. L., Snyder, A. Z., Morris, J. C., & Buckner, R. L. (2002). Under‐ recruitment and nonselective recruitment: dissociable neural mechanisms associated with aging. Neuron, 33(5), 827‐840. Lorente de No, R. (1938). Analysis of the activity of the chains of internuncial neurons. J. Neurophysiol., 1, 207‐244. Luebke, J., Barbas, H., & Peters, A. (2010). Effects of normal aging on prefrontal area 46 in the rhesus monkey. Brain Res Rev, 62(2), 212‐232. Luebke, J. I., Chang, Y. M., Moore, T. L., & Rosene, D. L. (2004). Normal aging results in decreased synaptic excitation and increased synaptic inhibition of layer 2/3 pyramidal cells in the monkey prefrontal cortex. Neuroscience, 125(1), 277‐288. Luppi, P. H., Aston‐Jones, G., Akaoka, H., Chouvet, G., & Jouvet, M. (1995). Afferent projections to the rat locus coeruleus demonstrated by retrograde and anterograde tracing with cholera‐toxin B subunit and Phaseolus vulgaris leucoagglutinin. Neuroscience, 65(1), 119‐160. Machamer, P., Darden, L., & Craver, C. F. (2000). Thinking about mechanisms. Philosophy of Science, 67, 1‐25. Mann, E. O., & Mody, I. (2010). Control of hippocampal gamma oscillation frequency by tonic inhibition and excitation of interneurons. Nat Neurosci, 13(2), 205‐212. Mansour, A., Fox, C. A., Burke, S., Meng, F., Thompson, R. C., Akil, H., et al. (1994). Mu, delta, and kappa opioid receptor mRNA expression in the rat CNS: an in situ hybridization study. J Comp Neurol, 350(3), 412‐438. 328 Marder, E., & Thirumalai, V. (2002). Cellular, synaptic and network effects of neuromodulation. Neural Netw, 15(4‐6), 479‐493. Markham, J. A., & Juraska, J. M. (2002). Aging and sex influence the anatomy of the rat anterior cingulate cortex. Neurobiol Aging, 23(4), 579‐588. Markram, H., Toledo‐Rodriguez, M., Wang, Y., Gupta, A., Silberberg, G., & Wu, C. (2004). Interneurons of the neocortical inhibitory system. Nat Rev Neurosci, 5(10), 793‐ 807. Marquis, J. P., Killcross, S., & Haddon, J. E. (2007). Inactivation of the prelimbic, but not infralimbic, prefrontal cortex impairs the contextual control of response conflict in rats. Eur J Neurosci, 25(2), 559‐566. Marr, D. (1969). A theory of cerebellar cortex. J Physiol, 202(2), 437‐470. Marr, D. (1970). A theory for cerebral neocortex. Proc R Soc Lond B Biol Sci, 176(43), 161‐234. Marr, D. (1971). Simple memory: a theory for archicortex. Philos Trans R Soc Lond B Biol Sci, 262(841), 23‐81. Mathis, A., Schunck, T., Erb, G., Namer, I. J., & Luthringer, R. (2009). The effect of aging on the inhibitory function in middle‐aged subjects: a functional MRI study coupled with a color‐matched Stroop task. Int J Geriatr Psychiatry, 24(10), 1062‐ 1071. Matsumoto, K., Suzuki, W., & Tanaka, K. (2003). Neuronal correlates of goal‐based motor selection in the prefrontal cortex. Science, 301(5630), 229‐232. Maviel, T., Durkin, T. P., Menzaghi, F., & Bontempi, B. (2004). Sites of neocortical reorganization critical for remote spatial memory. Science, 305(5680), 96‐99. McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5, 115‐133. McEchron, M. D., Bouwmeester, H., Tseng, W., Weiss, C., & Disterhoft, J. F. (1998). Hippocampectomy disrupts auditory trace fear conditioning and contextual fear conditioning in the rat. Hippocampus, 8(6), 638‐646. 329 Middleton, F. A., & Strick, P. L. (2000). Basal ganglia and cerebellar loops: motor and cognitive circuits. Brain Res Brain Res Rev, 31(2‐3), 236‐250. Milad, M. R., & Quirk, G. J. (2002). Neurons in medial prefrontal cortex signal memory for fear extinction. Nature, 420(6911), 70‐74. Milham, M. P., Erickson, K. I., Banich, M. T., Kramer, A. F., Webb, A., Wszalek, T., et al. (2002). Attentional control in the aging brain: insights from an fMRI study of the stroop task. Brain Cogn, 49(3), 277‐296. Miller, E. K., & Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. Annu Rev Neurosci, 24, 167‐202. Miller, E. K., Erickson, C. A., & Desimone, R. (1996). Neural mechanisms of visual working memory in prefrontal cortex of the macaque. J Neurosci, 16(16), 5154‐5167. Milner, B. (1963). Effects of different brain lesions on card sorting. Archives of Neurology, 9, 90‐100. Milner, B. (1982). Some cognitive effects of frontal‐lobe lesions in man. Philos Trans R Soc Lond B Biol Sci, 298(1089), 211‐226. Modirrousta, M., & Fellows, L. K. (2008). Dorsal medial prefrontal cortex plays a necessary role in rapid error prediction in humans. J Neurosci, 28(51), 14000‐ 14005. Modirrousta, M., & Fellows, L. K. (2008). Dorsal medial prefrontal cortex plays a necessary role in rapid error prediction in humans. J Neurosci, 28(51), 14000‐ 14005. Modirrousta, M., & Fellows, L. K. (2008). Medial prefrontal cortex plays a critical and selective role in 'feeling of knowing' meta‐memory judgments. Neuropsychologia, 46(12), 2958‐2965. Monchi, O., Petrides, M., Petre, V., Worsley, K., & Dagher, A. (2001). Wisconsin Card Sorting revisited: distinct neural circuits participating in different stages of the task identified by event‐related functional magnetic resonance imaging. J Neurosci, 21(19), 7733‐7741. Monckton, J. E., & McCormick, D. A. (2002). Neuromodulatory role of serotonin in the ferret thalamus. J Neurophysiol, 87(4), 2124‐2136. 330 Moore, T. L., Schettler, S. P., Killiany, R. J., Herndon, J. G., Luebke, J. I., Moss, M. B., et al. (2005). Cognitive impairment in aged rhesus monkeys associated with monoamine receptors in the prefrontal cortex. Behav Brain Res, 160(2), 208‐221. Morgan, M. A., & LeDoux, J. E. (1995). Differential contribution of dorsal and ventral medial prefrontal cortex to the acquisition and extinction of conditioned fear in rats. Behav Neurosci, 109(4), 681‐688. Morgan, M. A., Romanski, L. M., & LeDoux, J. E. (1993). Extinction of emotional learning: contribution of medial prefrontal cortex. Neurosci Lett, 163(1), 109‐113. Morris, R. G., Garrud, P., Rawlins, J. N., & O'Keefe, J. (1982). Place navigation impaired in rats with hippocampal lesions. Nature, 297(5868), 681‐683. Moscovitch, M., & Winocur, G. (1995). Frontal lobes, memory, and aging. Ann N Y Acad Sci, 769, 119‐150. Mountcastle, V. B. (1997). The columnar organization of the neocortex. Brain, 120 ( Pt 4), 701‐722. Mountcastle, V. B. (1997). The columnar organization of the neocortex. Brain, 120 ( Pt 4), 701‐722. Moyer, J. R., Jr., Deyo, R. A., & Disterhoft, J. F. (1990). Hippocampectomy disrupts trace eye‐blink conditioning in rabbits. Behav Neurosci, 104(2), 243‐252. Muenzinger, K. F. (1938). Vicarious trial and error at a point of choice: I. a general survey of its relation to learning efficiency. Journal of Genetic Psychology, 53, 75‐86. Mulder, A. B., Nordquist, R. E., Orgut, O., & Pennartz, C. M. (2003). Learning‐related changes in response patterns of prefrontal neurons during instrumental conditioning. Behav Brain Res, 146(1‐2), 77‐88. Myme, C. I., Sugino, K., Turrigiano, G. G., & Nelson, S. B. (2003). The NMDA‐to‐AMPA ratio at synapses onto layer 2/3 pyramidal neurons is conserved across prefrontal and visual cortices. J Neurophysiol, 90(2), 771‐779. Nadel, L., & Moscovitch, M. (1998). Hippocampal contributions to cortical plasticity. Neuropharmacology, 37(4‐5), 431‐439. 331 Nakahara, K., Hayashi, T., Konishi, S., & Miyashita, Y. (2002). Functional MRI of macaque monkeys performing a cognitive set‐shifting task. Science, 295(5559), 1532‐1536. Nakamura, M., Miyawaki, C., Matsushita, N., Yagi, R., & Handa, Y. (1998). Analysis of voluntary finger movements during hand tasks by a motion analyzer. J Electromyogr Kinesiol, 8(5), 295‐303. Neafsey, E. J., & Sievert, C. (1982). A second forelimb motor area exists in rat frontal cortex. Brain Res, 232(1), 151‐156. Neafsey, E. J., Terreberry, R. R., Hurley, K. M., Ruit, K. H., & Frysztak, R. J. (1993). Anterior cingulate cortex in rodents: connections, visceral control functions, and implications for emotion. In B. A. Vogt (Ed.), Neurobiology of cingulate cortex and limbic thalamus (pp. 206‐223). Boston: Birkhauser. Nicolle, M. M., & Baxter, M. G. (2003). Glutamate receptor binding in the frontal cortex and dorsal striatum of aged rats with impaired attentional set‐shifting. Eur J Neurosci, 18(12), 3335‐3342. Nieuwenhuis, S., Ridderinkhof, K. R., Talsma, D., Coles, M. G., Holroyd, C. B., Kok, A., et al. (2002). A computational account of altered error processing in older age: dopamine and the error‐related negativity. Cogn Affect Behav Neurosci, 2(1), 19‐ 36. Northcutt, R. G. (1981). Evolution of the telencephalon in nonmammals. Annu Rev Neurosci, 4, 301‐350. O'Reilly, R. C., & Frank, M. J. (2006). Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Comput, 18(2), 283‐328. Ostlund, S. B., Winterbauer, N. E., & Balleine, B. W. (2009). Evidence of action sequence chunking in goal‐directed instrumental conditioning and its dependence on the dorsomedial prefrontal cortex. J Neurosci, 29(25), 8280‐8287. Palomero‐Gallagher, N., & Zilles, K. (2004). Isocortex. In G. Paxinos (Ed.), The Rat Nervous System (pp. 728‐760). San Diego: Elsevier Academic Press. Pannu, J. K., & Kaszniak, A. W. (2005). Metamemory experiments in neurological populations: a review. Neuropsychol Rev, 15(3), 105‐130. 332 Pannu, J. K., Kaszniak, A. W., & Rapcsak, S. Z. (2005). Metamemory for faces following frontal lobe damage. J Int Neuropsychol Soc, 11(6), 668‐676. Pare, D., Quirk, G. J., & Ledoux, J. E. (2004). New vistas on amygdala networks in conditioned fear. J Neurophysiol, 92(1), 1‐9. Park, D. C., & Reuter‐Lorenz, P. (2009). The adaptive brain: aging and neurocognitive scaffolding. Annu Rev Psychol, 60, 173‐196. Passetti, F., Chudasama, Y., & Robbins, T. W. (2002). The frontal cortex of the rat and visual attentional performance: dissociable functions of distinct medial prefrontal subregions. Cereb Cortex, 12(12), 1254‐1268. Passetti, F., Chudasama, Y., & Robbins, T. W. (2002). The frontal cortex of the rat and visual attentional performance: dissociable functions of distinct medial prefrontal subregions. Cereb Cortex, 12(12), 1254‐1268. Paxton, J. L., Barch, D. M., Racine, C. A., & Braver, T. S. (2008). Cognitive control, goal maintenance, and prefrontal function in healthy aging. Cereb Cortex, 18(5), 1010‐1028. Peyrache, A., Khamassi, M., Benchenane, K., Wiener, S. I., & Battaglia, F. P. (2009). Replay of rule‐learning related neural patterns in the prefrontal cortex during sleep. Nat Neurosci, 12(7), 919‐926. Phillips, R. G., & LeDoux, J. E. (1992). Differential contribution of amygdala and hippocampus to cued and contextual fear conditioning. Behav Neurosci, 106(2), 274‐285. Pitkanen, A. (2000). Connectivity of the rat amygdaloid complex. In J. P. Aggleton (Ed.), The Amygdala: A Functional Analysis (2nd ed., pp. 31‐116). Oxford: Oxford University Press. Pratt, W. E., & Mizumori, S. J. (2001). Neurons in rat medial prefrontal cortex show anticipatory rate changes to predictable differential rewards in a spatial memory task. Behav Brain Res, 123(2), 165‐183. Preuss, T. M. (1995). Do rats have prefrontal cortex? The Rose‐Woolsey‐Akert program reconsidered. Journal of Cognitive Neuroscience, 7(1), 1‐24. 333 Preuss, T. M. (2000). Taking the measure of diversity: comparative alternatives to the model‐animal paradigm in cortical neuroscience. Brain Behav Evol, 55(6), 287‐ 299. Qiu, Y. H., Wu, X. Y., Xu, H., & Sackett, D. (2009). Neuroimaging study of placebo analgesia in humans. Neurosci Bull, 25(5), 277‐282. Quilodran, R., Rothe, M., & Procyk, E. (2008). Behavioral shifts and action valuation in the anterior cingulate cortex. Neuron, 57(2), 314‐325. Quinn, J. J., Ma, Q. D., Tinsley, M. R., Koch, C., & Fanselow, M. S. (2008). Inverse temporal contributions of the dorsal hippocampus and medial prefrontal cortex to the expression of long‐term fear memories. Learn Mem, 15(5), 368‐372. Rabbitt, P. M. A. (1968). Age and the use of structure in transmitted information. In G. A. Talland (Ed.), Human Aging and Behavior. New York: Academic Press. Ragozzino, M. E., Adams, S., & Kesner, R. P. (1998). Differential involvement of the dorsal anterior cingulate and prelimbic‐infralimbic areas of the rodent prefrontal cortex in spatial working memory. Behav Neurosci, 112(2), 293‐303. Ragozzino, M. E., Detrick, S., & Kesner, R. P. (1999). Involvement of the prelimbic‐ infralimbic areas of the rodent prefrontal cortex in behavioral flexibility for place and response learning. J Neurosci, 19(11), 4585‐4594. Ragozzino, M. E., Detrick, S., & Kesner, R. P. (2002). The effects of prelimbic and infralimbic lesions on working memory for visual objects in rats. Neurobiol Learn Mem, 77(1), 29‐43. Ragozzino, M. E., & Kesner, R. P. (2001). The role of rat dorsomedial prefrontal cortex in working memory for egocentric responses. Neurosci Lett, 308(3), 145‐148. Ragozzino, M. E., Kim, J., Hassert, D., Minniti, N., & Kiang, C. (2003). The contribution of the rat prelimbic‐infralimbic areas to different forms of task switching. Behav Neurosci, 117(5), 1054‐1065. Ragozzino, M. E., Wilcox, C., Raso, M., & Kesner, R. P. (1999). Involvement of rodent prefrontal cortex subregions in strategy switching. Behav Neurosci, 113(1), 32‐ 41. 334 Rasch, M., Logothetis, N. K., & Kreiman, G. (2009). From neurons to circuits: linear estimation of local field potentials. J Neurosci, 29(44), 13785‐13796. Raz, N. (2000). Aging of the brain and its impact on cognitive performance: integration of structural and functional findings. In F. I. M. Craik & T. A. Salthouse (Eds.), The Handbook of Aging and Cognition (pp. 1‐90). Mahwah: Lawrence Erlbaum Associates, Inc. Raz, N., & Rodrigue, K. M. (2006). Differential aging of the brain: patterns, cognitive correlates and modifiers. Neurosci Biobehav Rev, 30(6), 730‐748. Recanzone, G. H., & Sutter, M. L. (2008). The biological basis of audition. Annu Rev Psychol, 59, 119‐142. Renart, A., de la Rocha, J., Bartho, P., Hollender, L., Parga, N., Reyes, A., et al. (2010). The asynchronous state in cortical circuits. Science, 327(5965), 587‐590. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. New York: Appleton‐ Century‐Crofts. Rich, E. L., & Shapiro, M. (2009). Rat prefrontal cortical neurons selectively code strategy switches. J Neurosci, 29(22), 7208‐7219. Rich, E. L., & Shapiro, M. L. (2007). Prelimbic/infralimbic inactivation impairs memory for multiple task switches, but not flexible selection of familiar tasks. J Neurosci, 27(17), 4747‐4755. Riedel, G., Micheau, J., Lam, A. G., Roloff, E. L., Martin, S. J., Bridge, H., et al. (1999). Reversible neural inactivation reveals hippocampal participation in several memory processes. Nat Neurosci, 2(10), 898‐905. Risold, P. Y., & Swanson, L. W. (1997). Connections of the rat lateral septal complex. Brain Res Brain Res Rev, 24(2‐3), 115‐195. Robbins, T. W. (2005). Chemistry of the mind: neurochemical modulation of prefrontal cortical function. J Comp Neurol, 493(1), 140‐146. Rochester, N., Holland, J. H., Haibt, L. H., & Duda, W. L. (1956). Tests on a cell assembly theory of the action of the brain, using a large digital computer. IRE Transactions on Information Theory, 2, 80‐93. 335 Rose, J. E., & Woolsey, C. N. (1948). Structure and relations of limbic cortex and anterior thalamic nuclei in rabbit and cat. J Comp Neurol, 89(3), 279‐347. Rosenkranz, J. A., & Grace, A. A. (2001). Dopamine attenuates prefrontal cortical suppression of sensory inputs to the basolateral amygdala of rats. J Neurosci, 21(11), 4090‐4103. Rudebeck, P. H., Walton, M. E., Smyth, A. N., Bannerman, D. M., & Rushworth, M. F. (2006). Separate neural pathways process different decision costs. Nat Neurosci, 9(9), 1161‐1168. Rushworth, M. F., & Behrens, T. E. (2008). Choice, uncertainty and value in prefrontal and cingulate cortex. Nat Neurosci, 11(4), 389‐397. Ryle, G. (1949). The Concept of Mind. Chicago: University of Chicago Press. Sakurai, Y., & Sugimoto, S. (1986). Multiple unit activity of prefrontal cortex and dorsomedial thalamus during delayed go/no‐go alternation in the rat. Behav Brain Res, 20(3), 295‐301. Sallet, J., Quilodran, R., Rothe, M., Vezoli, J., Joseph, J. P., & Procyk, E. (2007). Expectations, gains, and losses in the anterior cingulate cortex. Cogn Affect Behav Neurosci, 7(4), 327‐336. Santana, N., Bortolozzi, A., Serrats, J., Mengod, G., & Artigas, F. (2004). Expression of serotonin1A and serotonin2A receptors in pyramidal and GABAergic neurons of the rat prefrontal cortex. Cereb Cortex, 14(10), 1100‐1109. Santana, N., Mengod, G., & Artigas, F. (2009). Quantitative analysis of the expression of dopamine D1 and D2 receptors in pyramidal and GABAergic neurons of the rat prefrontal cortex. Cereb Cortex, 19(4), 849‐860. Sara, S. J., & Herve‐Minvielle, A. (1995). Inhibitory influence of frontal cortex on locus coeruleus neurons. Proc Natl Acad Sci U S A, 92(13), 6032‐6036. Sarter, M., & Bruno, J. P. (2004). Developmental origins of the age‐related decline in cortical cholinergic function and associated cognitive abilities. Neurobiol Aging, 25(9), 1127‐1139. 336 Scherzer, C. R., Landwehrmeyer, G. B., Kerner, J. A., Counihan, T. J., Kosinski, C. M., Standaert, D. G., et al. (1998). Expression of N‐methyl‐D‐aspartate receptor subunit mRNAs in the human brain: hippocampus and cortex. J Comp Neurol, 390(1), 75‐90. Schnyer, D. M., Verfaellie, M., Alexander, M. P., LaFleche, G., Nicholls, L., & Kaszniak, A. W. (2004). A role for right medial prefontal cortex in accurate feeling‐of‐knowing judgements: evidence from patients with lesions to frontal cortex. Neuropsychologia, 42(7), 957‐966. Schnyer, D. M., Verfaellie, M., Alexander, M. P., LaFleche, G., Nicholls, L., & Kaszniak, A. W. (2004). A role for right medial prefontal cortex in accurate feeling‐of‐knowing judgements: evidence from patients with lesions to frontal cortex. Neuropsychologia, 42(7), 957‐966. Schoenbaum, G., Nugent, S. L., Saddoris, M. P., & Setlow, B. (2002). Orbitofrontal lesions in rats impair reversal but not acquisition of go, no‐go odor discriminations. Neuroreport, 13(6), 885‐890. Schultz, W. (2002). Getting formal with dopamine and reward. Neuron, 36(2), 241‐263. Schweimer, J., & Hauber, W. (2005). Involvement of the rat anterior cingulate cortex in control of instrumental responses guided by reward expectancy. Learn Mem, 12(3), 334‐342. Scoville, W. B., & Milner, B. (1957). Loss of recent memory after bilateral hippocampal lesions. J Neurol Neurosurg Psychiatry, 20(1), 11‐21. Seamans, J. K., Floresco, S. B., & Phillips, A. G. (1995). Functional differences between the prelimbic and anterior cingulate regions of the rat prefrontal cortex. Behav Neurosci, 109(6), 1063‐1073. Sesack, S. R., Deutch, A. Y., Roth, R. H., & Bunney, B. S. (1989). Topographical organization of the efferent projections of the medial prefrontal cortex in the rat: an anterograde tract‐tracing study with Phaseolus vulgaris leucoagglutinin. J Comp Neurol, 290(2), 213‐242. Sesack, S. R., & Grace, A. A. (2010). Cortico‐Basal Ganglia reward network: microcircuitry. Neuropsychopharmacology, 35(1), 27‐47. 337 Sheer, D. E., & Grandstaff, N. (1970). Computer‐analysis of electrical activity in the brain and its relation to behavior. Bibl Psychiatr, 143, 160‐172. Sherrington, S. C. (1906). The Integrative Action of the Nervous System (2nd, 1947 ed.). New Haven: Yale University Press. Shettleworth, S. J. (2009). The evolution of comparative cognition: is the snark still a boojum? Behav Processes, 80(3), 210‐217. Shettleworth, S. J. (2010). Cognition, Evolution, and Behavior. New York: Oxford University Press. Shidara, M., & Richmond, B. J. (2002). Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science, 296(5573), 1709‐1711. Shima, K., Isoda, M., Mushiake, H., & Tanji, J. (2007). Categorization of behavioural sequences in the prefrontal cortex. Nature, 445(7125), 315‐318. Shima, K., & Tanji, J. (2000). Neuronal activity in the supplementary and presupplementary motor areas for temporal organization of multiple movements. J Neurophysiol, 84(4), 2148‐2160. Shu, Y., Hasenstaub, A., & McCormick, D. A. (2003). Turning on and off recurrent balanced cortical activity. Nature, 423(6937), 288‐293. Siapas, A. G., Lubenov, E. V., & Wilson, M. A. (2005). Prefrontal phase locking to hippocampal theta oscillations. Neuron, 46(1), 141‐151. Sirota, A., Montgomery, S., Fujisawa, S., Isomura, Y., Zugaro, M., & Buzsaki, G. (2008). Entrainment of neocortical neurons and gamma oscillations by the hippocampal theta rhythm. Neuron, 60(4), 683‐697. Skaggs, W. E., Knierim, J. J., Kudrimoti, H. S., & McNaughton, B. L. (1995). A model of the neural basis of the rat's sense of direction. Adv Neural Inf Process Syst, 7, 173‐ 180. Smith, D. E., Rapp, P. R., McKay, H. M., Roberts, J. A., & Tuszynski, M. H. (2004). Memory impairment in aged primates is associated with focal death of cortical neurons and atrophy of subcortical neurons. J Neurosci, 24(18), 4373‐4381. 338 Smith, Y., Pare, J. F., & Pare, D. (2000). Differential innervation of parvalbumin‐ immunoreactive interneurons of the basolateral amygdaloid complex by cortical and intrinsic inputs. J Comp Neurol, 416(4), 496‐508. Solomon, P. R., Vander Schaaf, E. R., Thompson, R. F., & Weisz, D. J. (1986). Hippocampus and trace conditioning of the rabbit's classically conditioned nictitating membrane response. Behav Neurosci, 100(5), 729‐744. Sotres‐Bayon, F., & Quirk, G. J. (2010). Prefrontal control of fear: more than just extinction. Curr Opin Neurobiol, 20(2), 231‐235. Squire, L. R. (1992). Memory and the hippocampus: a synthesis from findings with rats, monkeys, and humans. Psychol Rev, 99(2), 195‐231. St Onge, J. R., & Floresco, S. B. (2009). Prefrontal cortical contribution to risk‐based decision making. Cereb Cortex, 20(8), 1816‐1828. Stephens, D. N., Weidmann, R., Quartermain, D., & Sarter, M. (1985). Reversal learning in senescent rats. Behav Brain Res, 17(3), 193‐202. Stern, Y., Zarahn, E., Hilton, H. J., Flynn, J., DeLaPaz, R., & Rakitin, B. (2003). Exploring the neural basis of cognitive reserve. J Clin Exp Neuropsychol, 25(5), 691‐701. Strong, R., Hicks, P., Hsu, L., Bartus, R. T., & Enna, S. J. (1980). Age‐related alterations in the rodent brain cholinergic system and behavior. Neurobiol Aging, 1(1), 59‐63. Stumpf, C. (1965). The Fast Component in the Electrical Activity of Rabbit's Hippocampus. Electroencephalogr Clin Neurophysiol, 18, 477‐486. Sur, M., & Leamey, C. A. (2001). Development and plasticity of cortical areas and networks. Nat Rev Neurosci, 2(4), 251‐262. Swanson, L. W. (1981). A direct projection from Ammon's horn to prefrontal cortex in the rat. Brain Res, 217(1), 150‐154. Takehara, K., Kawahara, S., & Kirino, Y. (2003). Time‐dependent reorganization of the brain components underlying memory retention in trace eyeblink conditioning. J Neurosci, 23(30), 9897‐9905. Takehara, K., Kawahara, S., Munemoto, Y., Kuriyama, H., Mori, H., Mishina, M., et al. (2004). The N‐methyl‐D‐aspartate (NMDA)‐type glutamate receptor 339 GluRepsilon2 is important for delay and trace eyeblink conditioning in mice. Neurosci Lett, 364(1), 43‐47. Takehara‐Nishiuchi, K., Kawahara, S., & Kirino, Y. (2005). NMDA receptor‐dependent processes in the medial prefrontal cortex are important for acquisition and the early stage of consolidation during trace, but not delay eyeblink conditioning. Learn Mem, 12(6), 606‐614. Takehara‐Nishiuchi, K., & McNaughton, B. L. (2008). Spontaneous changes of neocortical code for associative memory during consolidation. Science, 322(5903), 960‐963. Takehara‐Nishiuchi, K., Nakao, K., Kawahara, S., Matsuki, N., & Kirino, Y. (2006). Systems consolidation requires postlearning activation of NMDA receptors in the medial prefrontal cortex in trace eyeblink conditioning. J Neurosci, 26(19), 5049‐5058. Tanaka, S. C., Balleine, B. W., & O'Doherty, J. P. (2008). Calculating consequences: brain systems that encode the causal effects of actions. J Neurosci, 28(26), 6750‐6755. Tanji, J. (2001). Sequential organization of multiple movements: involvement of cortical motor areas. Annu Rev Neurosci, 24, 631‐651. Tanji, J., & Shima, K. (1994). Role for supplementary motor area cells in planning several movements ahead. Nature, 371(6496), 413‐416. Teixeira, C. M., Pomedli, S. R., Maei, H. R., Kee, N., & Frankland, P. W. (2006). Involvement of the anterior cingulate cortex in the expression of remote spatial memory. J Neurosci, 26(29), 7555‐7564. Tiesinga, P., & Sejnowski, T. J. (2009). Cortical enlightenment: are attentional gamma oscillations driven by ING or PING? Neuron, 63(6), 727‐732. Tiesinga, P. H., Fellous, J. M., Salinas, E., Jose, J. V., & Sejnowski, T. J. (2004). Synchronization as a mechanism for attentional gain modulation. Neurocomputing, 58‐60, 641‐646. Tiesinga, P. H., & Sejnowski, T. J. (2001). Precision of pulse‐coupled networks of integrate‐and‐fire neurons. Network, 12(2), 215‐233. Todman, D. (2008). John Eccles (1903‐97) and the experiment that proved chemical synaptic transmission in the central nervous system. J Clin Neurosci, 15(9), 972‐ 977. 340 Tolman, E. C. (1948). Cognitive maps in rats and men. The Psychological Review, 55(4), 189‐208. Townsend, J., Adamo, M., & Haist, F. (2006). Changing channels: an fMRI study of aging and cross‐modal attention shifts. Neuroimage, 31(4), 1682‐1692. Traub, R. D., Jefferys, J. G., & Whittington, M. A. (1997). Simulation of gamma rhythms in networks of interneurons and pyramidal cells. J Comput Neurosci, 4(2), 141‐ 150. Traub, R. D., Whittington, M. A., Colling, S. B., Buzsaki, G., & Jefferys, J. G. (1996). Analysis of gamma rhythms in the rat hippocampus in vitro and in vivo. J Physiol, 493 ( Pt 2), 471‐484. Traub, R. D., Whittington, M. A., Stanford, I. M., & Jefferys, J. G. (1996). A mechanism for generation of long‐range synchronous fast oscillations in the cortex. Nature, 383(6601), 621‐624. Tremblay, S., Saint‐Aubin, J., & Jalbert, A. (2006). Rehearsal in serial memory for visual‐ spatial information: evidence from eye movements. Psychon Bull Rev, 13(3), 452‐ 457. Trujillo, L. T., & Allen, J. J. (2007). Theta EEG dynamics of the error‐related negativity. Clin Neurophysiol, 118(3), 645‐668. Tuvnes, F. A., Steffenach, H. A., Murison, R., Moser, M. B., & Moser, E. I. (2003). Selective hippocampal lesions do not increase adrenocortical activity. J Neurosci, 23(10), 4345‐4354. Tzschentke, T. M. (2001). Pharmacology and behavioral pharmacology of the mesocortical dopamine system. Prog Neurobiol, 63(3), 241‐320. Tzschentke, T. M., & Schmidt, W. J. (1999). Functional heterogeneity of the rat medial prefrontal cortex: effects of discrete subarea‐specific lesions on drug‐induced conditioned place preference and behavioural sensitization. Eur J Neurosci, 11(11), 4099‐4109. Ungerleider, L. G., & Mishkin, M. (1982). Two cortical visual systems. In D. I. Ingle, M. A. Goodale & R. J. W. Mansfield (Eds.), Analysis of Visual Behavior (pp. 549‐586). Cambridge: MIT Press. 341 Vaidya, J. G., Paradiso, S., Boles Ponto, L. L., McCormick, L. M., & Robinson, R. G. (2007). Aging, grey matter, and blood flow in the anterior cingulate cortex. Neuroimage, 37(4), 1346‐1353. van der Meer, M. A., Johnson, A., Schmitzer‐Torbert, N. C., & Redish, A. D. (2010). Triple dissociation of information processing in dorsal striatum, ventral striatum, and hippocampus on a learned spatial decision task. Neuron, 67(1), 25‐32. van der Meer, M. A., & Redish, A. D. (2009). Covert Expectation‐of‐Reward in Rat Ventral Striatum at Decision Points. Front Integr Neurosci, 3, 1. van Groen, T., Kadish, I., Popovic, N., Popovic, M., Caballero‐Bleda, M., Bano‐Otalora, B., et al. (2009). Age‐related brain pathology in Octodon degu: Blood vessel, white matter and Alzheimer‐like pathology. Neurobiol Aging. Van Hoesen, G. W., Morecraft, R. J., & Vogt, B. A. (1993). Connections of the monkey cingulate cortex. In B. A. Vogt & M. Gabriel (Eds.), Neurobiology of cingulate cortex and limbic thalamus (pp. 249‐284). Boston: Birkhauser. van Veen, V., & Carter, C. S. (2002). The anterior cingulate as a conflict monitor: fMRI and ERP studies. Physiol Behav, 77(4‐5), 477‐482. Vann, S. D., & Aggleton, J. P. (2004). The mammillary bodies: two memory systems in one? Nat Rev Neurosci, 5(1), 35‐44. Verhaeghen, P., & De Meersman, L. (1998). Aging and the Stroop effect: a meta‐analysis. Psychol Aging, 13(1), 120‐126. Vertes, R. P. (2004). Differential projections of the infralimbic and prelimbic cortex in the rat. Synapse, 51(1), 32‐58. Vogt, B. A. (1993). Structure organization of cingulate cortex: areas, neurons, and somatodendritic transmitter receptors. In B. A. Vogt & M. Gabriel (Eds.), Neurobiology of cingulate cortex and limbic thalamus (pp. 19‐70). Boston: Birkhauser. Vogt, B. A., Finch, D. M., & Olson, C. R. (1992). Functional heterogeneity in cingulate cortex: the anterior executive and posterior evaluative regions. Cereb Cortex, 2(6), 435‐443. 342 Vogt, B. A., & Peters, A. (1981). Form and distribution of neurons in rat cingulate cortex: areas 32, 24, and 29. J Comp Neurol, 195(4), 603‐625. Vogt, B. A., Rosene, D. L., & Peters, A. (1981). Synaptic termination of thalamic and callosal afferents in cingulate cortex of the rat. J Comp Neurol, 201(2), 265‐283. Vogt, B. A., Vogt, L., & Farber, N. B. (2004). Cingulate cortex and disease models. In G. Paxinos (Ed.), The Rat Nervous System (pp. 704‐727). San Diego: Elsevier Academic Press. Voorn, P., Vanderschuren, L. J., Groenewegen, H. J., Robbins, T. W., & Pennartz, C. M. (2004). Putting a spin on the dorsal‐ventral divide of the striatum. Trends Neurosci, 27(8), 468‐474. Vysokanov, A., Flores‐Hernandez, J., & Surmeier, D. J. (1998). mRNAs for clozapine‐ sensitive receptors co‐localize in rat prefrontal cortex neurons. Neurosci Lett, 258(3), 179‐182. Walton, M. E., Bannerman, D. M., Alterescu, K., & Rushworth, M. F. (2003). Functional specialization within medial frontal cortex of the anterior cingulate for evaluating effort‐related decisions. J Neurosci, 23(16), 6475‐6479. Walton, M. E., Bannerman, D. M., & Rushworth, M. F. (2002). The role of rat medial frontal cortex in effort‐based decision making. J Neurosci, 22(24), 10996‐11003. Walton, M. E., Groves, J., Jennings, K. A., Croxson, P. L., Sharp, T., Rushworth, M. F., et al. (2009). Comparing the role of the anterior cingulate cortex and 6‐ hydroxydopamine nucleus accumbens lesions on operant effort‐based decision making. Eur J Neurosci, 29(8), 1678‐1691. Wang, X., Zhong, P., Gu, Z., & Yan, Z. (2003). Regulation of NMDA receptors by dopamine D4 signaling in prefrontal cortex. J Neurosci, 23(30), 9852‐9861. Wang, X. J. (1999). Synaptic basis of cortical persistent activity: the importance of NMDA receptors to working memory. J Neurosci, 19(21), 9587‐9603. Wang, X. J., & Buzsaki, G. (1996). Gamma oscillation by synaptic inhibition in a hippocampal interneuronal network model. J Neurosci, 16(20), 6402‐6413. Watson, T. C., Jones, M. W., & Apps, R. (2009). Electrophysiological mapping of novel prefrontal ‐ cerebellar pathways. Front Integr Neurosci, 3, 18. 343 Weiss, C., Bouwmeester, H., Power, J. M., & Disterhoft, J. F. (1999). Hippocampal lesions prevent trace eyeblink conditioning in the freely moving rat. Behav Brain Res, 99, 123‐132. Welsh, D. K., Takahashi, J. S., & Kay, S. A. (2010). Suprachiasmatic nucleus: cell autonomy and network properties. Annu Rev Physiol, 72, 551‐577. West, R. (2004). The effects of aging on controlled attention and conflict processing in the Stroop task. J Cogn Neurosci, 16(1), 103‐113. West, R., & Alain, C. (1999). Event‐related neural activity associated with the Stroop task. Brain Res Cogn Brain Res, 8(2), 157‐164. West, R., & Alain, C. (2000). Age‐related decline in inhibitory control contributes to the increased Stroop effect observed in older adults. Psychophysiology, 37(2), 179‐ 189. West, R., & Baylis, G. C. (1998). Effects of increased response dominance and contextual disintegration on the Stroop interference effect in older adults. Psychol Aging, 13(2), 206‐217. Whalley, L. J., Deary, I. J., Appleton, C. L., & Starr, J. M. (2004). Cognitive reserve and the neurobiology of cognitive aging. Ageing Res Rev, 3(4), 369‐382. Whittington, M. A., Traub, R. D., & Jefferys, J. G. (1995). Synchronized oscillations in interneuron networks driven by metabotropic glutamate receptor activation. Nature, 373(6515), 612‐615. Whittington, M. A., Traub, R. D., Kopell, N., Ermentrout, B., & Buhl, E. H. (2000). Inhibition‐based rhythms: experimental and mathematical observations on network dynamics. Int J Psychophysiol, 38(3), 315‐336. Williams, G. V., & Goldman‐Rakic, P. S. (1995). Modulation of memory fields by dopamine D1 receptors in prefrontal cortex. Nature, 376(6541), 572‐575. Wilson, H. R., & Cowan, J. D. (1972). Excitatory and inhibitory interactions in localized populations of model neurons. Biophys J, 12(1), 1‐24. 344 Winocur, G., Moscovitch, M., Fogel, S., Rosenbaum, R. S., & Sekeres, M. (2005). Preserved spatial memory after hippocampal lesions: effects of extensive experience in a complex environment. Nat Neurosci, 8(3), 273‐275. Winocur, G., Moscovitch, M., & Sekeres, M. (2007). Memory consolidation or transformation: context manipulation and hippocampal representations of memory. Nat Neurosci, 10(5), 555‐557. Winslow, J. T., & Insel, T. R. (1990). Serotonergic and catecholaminergic reuptake inhibitors have opposite effects on the ultrasonic isolation calls of rat pups. Neuropsychopharmacology, 3(1), 51‐59. Witter, M. P., & Amaral, D. G. (2004). Hippocampal formation. In G. Paxinos (Ed.), The Rat Nervous System (pp. 637‐703). San Diego: Elsevier Academic Press. Womelsdorf, T., Schoffelen, J. M., Oostenveld, R., Singer, W., Desimone, R., Engel, A. K., et al. (2007). Modulation of neuronal interactions through neuronal synchronization. Science, 316(5831), 1609‐1612. Wyss, J. M., & Sripanidkulchai, K. (1984). The topography of the mesencephalic and pontine projections from the cingulate cortex of the rat. Brain Res, 293(1), 1‐15. Yates, M. A., Markham, J. A., Anderson, S. E., Morris, J. R., & Juraska, J. M. (2008). Regional variability in age‐related loss of neurons from the primary visual cortex and medial prefrontal cortex of male and female rats. Brain Res, 1218, 1‐12. Yeung, N., Botvinick, M. M., & Cohen, J. D. (2004). The neural basis of error detection: conflict monitoring and the error‐related negativity. Psychol Rev, 111(4), 931‐ 959. Young, J. J., & Shapiro, M. L. (2009). Double dissociation and hierarchical organization of strategy switches and reversals in the rat PFC. Behav Neurosci, 123(5), 1028‐ 1035. Young, L. J., & Wang, Z. (2004). The neurobiology of pair bonding. Nat Neurosci, 7(10), 1048‐1054. Young, M. P., Tanaka, K., & Yamane, S. (1992). On oscillating neuronal responses in the visual cortex of the monkey. J Neurophysiol, 67(6), 1464‐1474. 345 Zahm, D. S. (1999). Functional‐anatomical implications of the nucleus accumbens core and shell subterritories. Ann N Y Acad Sci, 877, 113‐128. Zubieta, J. K., & Stohler, C. S. (2009). Neurobiological mechanisms of placebo responses. Ann N Y Acad Sci, 1156, 198‐210. Zysset, S., Schroeter, M. L., Neumann, J., & Yves von Cramon, D. (2006). Stroop interference, hemodynamic response and aging: An event‐related fMRI study. Neurobiol Aging.
* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project