IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 51, NO. 4, APRIL 2004 657 Optic Nerve Signals in a Neuromorphic Chip I: Outer and Inner Retina Models Kareem A. Zaghloul, Member, IEEE, and Kwabena Boahen* Abstract—We present a novel model for the mammalian retina and analyze its behavior. Our outer retina model performs bandpass spatiotemporal filtering. It is comprised of two reciprocally connected resistive grids that model the cone and horizontal cell syncytia. We show analytically that its sensitivity is proportional to the space-constant-ratio of the two grids while its half-max response is set by the local average intensity. Thus, this outer retina model realizes luminance adaptation. Our inner retina model performs high-pass temporal filtering. It features slow negative feedback whose strength is modulated by a locally computed measure of temporal contrast, modeling two kinds of amacrine cells, one narrow-field, the other wide-field. We show analytically that, when the input is spectrally pure, the corner-frequency tracks the input frequency. But when the input is broadband, the corner frequency is proportional to contrast. Thus, this inner retina model realizes temporal frequency adaptation as well as contrast gain control. We present CMOS circuit designs for our retina model in this paper as well. Experimental measurements from the fabricated chip, and validation of our analytical results, are presented in the companion paper [Zaghloul and Boahen (2004)]. Index Terms—Adaptive circuits, neural systems, neuromorphic engineering, prosthetics, vision. I. RETINOMORPHIC SYSTEMS T HE RETINA, one of the best studied neural systems, is a complex piece of biological wetware designed to signal the onset or offset of visual stimuli in a sustained or transient fashion . To encode these signals into spike patterns for transmission to higher processing centers, the retina has evolved intricate neuronal circuits that capture information contained within natural scenes efficiently . This visual preprocessing, realized by the retina, occurs in two stages, the outer retina and the inner retina. Each local retinal microcircuit plays a specific role in the retina’s function, and neurophysiologists have extracted a wealth of data characterizing how its constituent cell types contribute to visual processing. These physiological functions can be replicated in artificial systems by emulating their underlying synaptic interactions. In this paper, we present a novel model for how the outer and inner retina process visual Manuscript received September 6, 2002; revised June 25, 2003 This work was supported in part by the National Institutes of Health (NIH) under Vision Training Grant T32-EY07035 and in part by the Whitaker Foundation under Grant 37005-00-00. The work of K. A. Zaghloul was completed while he was with the Department of Neuroscience, University of Pennsylvania, Philadelphia, PA 19104 USA. Asterisk indicates corresponding author. K. A. Zaghloul is with the Department of Neurosurgery, University of Pennsylvania, Philadelphia, PA 19104 USA (e-mail: [email protected]). *K. Boahen is with the Department of Bioengineering, 120 Hayden Hall, 3320 Smith Walk, University of Pennsylvania, Philadelphia, PA 19104 USA (e-mail: [email protected]). Digital Object Identifier 10.1109/TBME.2003.821039 information, implement this model in silicon, and analyze its behavior. In the accompanying paper , we characterize how our silicon chip processes visual information and validate our analytical results. Attempts have been made to duplicate neural function at high levels of abstraction because detailed knowledge of neural circuits was unavailable, but these neuro-inspired systems proved ill-suited to direct hardware implementation because of the mathematical operations they require. Neuromorphic systems, on the other hand, which are constructed from neuron-like elements based on physiology, connected together into local microcircuits based on anatomy, can now duplicate certain brain computations in silicon , . The advantage these neuromorphic chips offer over software models is their ability to replicate neural computations in real-time at low power. This implementation efficiency translates to a greater ability to explore model parameters to further understand the underlying biological system and, by communicating with other neuromorphic chips, a greater ability to replicate more complicated neural systems. Furthermore, with the continued miniaturization of silicon technology and with a greater understanding of neural computations, attention has recently been shifted to developing prosthetic devices capable of restoring neural function , . While these efforts have been relatively successful in the cochlea, fully implantable solutions remain elusive . Neuroprostheses, such as an intraocular retinal implant, may benefit from a fully integrated approach made possible by the greater autonomy and functionality of neuromorphic systems. The most recent effort to model retinal processing in silicon incorporated outer retina circuitry as well as bipolar cell (BC) and amacrine cell interactions in the inner retina . This outer retina circuit took the difference between the photoreceptor signal and its spatiotemporal average, computed by a network of coupled lateral elements [horizontal cells (HCs)], through negative (inhibitory) feedback. Cone-coupling in this model attenuated high-frequency noise to realize a spatial bandpass filter and dynamic range was extended by implementing local automatic gain control. Furthermore, this model used HC activity to boost cone to HC excitation , which eliminated receptive-field expansion and temporal instability. This gain boosting mechanism has some physiological basis since glutamate release from cones is modulated by HC hemichannels , and may be enhanced further by HC autofeedback . Our outer retina design is based on this approach . The BC and amacrine cell interactions introduced by this earlier design represented a model for inner retina processing that, much like the circuit discussed here, attempted to capture 0018-9294/04$20.00 © 2004 IEEE 658 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 51, NO. 4, APRIL 2004 b) Cone Outer Segments Cone Terminals Horizontal Cells Excitation Inhibition CO c) +HC 1 Conduction Modulation HC HC Response a) 1 CT CT Spatial Frequency Fig. 1. Outer retina model. (a) Neural circuit: CTs receive a signal that is proportional to incident light intensity from the cone outer segment (CO) and excite HCs). HCs spread their input laterally through gap junctions, provide shunting inhibition onto CT, and modulate cone coupling and cone excitation. (b) System diagram: Signals travel from CO to CT and on to HC, which provides negative feedback. Excitation of HC by CT is modulated by HC, which also modulates the attenuation from CO to CT. These interactions realize local automatic gain control in CT and keep receptive field size invariant. Both CT and HC form networks, connected through gap junctions, that are governed by their respective space constants, l and l . (c) Frequency responses: Both HC and CT low-pass filter input signals, but because of HCs larger space constant, l , HC inhibition eliminates low-frequency signals, yielding a spatially bandpass response in CT. temporal adaptation through adjustment of amacrine cell feedback inhibition. However, this earlier design did not include the retina’s complementary push-pull architecture . Hence, at low-frequency stimulation, dc current levels tended to rise and, thus, modulation of the feedback loop gain was not realized as intended . Furthermore, this previous implementation did not replicate synaptic interactions at the ganglion cell (GC) level and, thus, did not faithfully capture the behavior of the retina’s major output pathways . In this paper, we present a novel model for processing in the mammalian retina that addresses the design flaws in  and extends that model to include GC level synaptic interactions. Our model is based on the functional architecture of the mammalian retina, and on physiological studies that have characterized the computations performed by the retina. Additionally, we build upon these earlier studies to present new circuit components that may account for some of the computations found in the retina. While the ultimate goal of our modeling effort is to match the mammalian retina, a detailed comparison between our model’s outputs and the retina’s outputs will be presented in a forthcoming paper. The present paper presents a detailed analysis of our mathematical model and describes its silicon implementation. The remainder of this paper is organized as follows. In Section II, we present our outer retina ciruit model, briefly revisiting a circuit design described earlier , and our model of BC circuitry that divides signals into ON and OFF channels. In Section III, we introduce a novel model for the inner retina that adjusts its loop gain, allowing it to adapt to different input power spectra. Finally, in Section IV, we discuss some of the assumptions underlying our analysis and justifications for our model. Section V concludes the paper. An accompanying paper presents experimental results that characterize our model’s behavior. II. OUTER RETINA MODEL Our model for the outer retina’s synaptic interactions, which realize spatiotemporal bandpass filtering and local gain control, was described previously . The synaptic interactions underlying our model are repeated in the Fig. 1(a) for reference. Briefly, photons incident on the cone (CO) reduce current flow to its cone terminal (CT), causing a hyperpolarization and a decrease in neurotransmitter release. CTs excite HCs which apply shunting (or attenuating) inhibition on to CT . This reciprocal interaction between cones and HCs, and electrical coupling of both cell types to their neighbors through gap junctions , creates a spatiotemporally bandpassed signal at CT that adapts to light intensity . From the synaptic interactions, we derived the block diagram shown in Fig. 1(b) by modeling both the cone and HC networks as spatial low-pass filters. We can derive the system level equations that describe how CO determines HC and CT activity, repand respectively, from this block diagram. resented by These equations, reproduced from , are (1) (2) where and are the cone- and horizontal-network space-conis the attenuation from CO to CT, and stants, respectively, is the amplification from CT to HC. HCs have stronger coupling in our model (i.e., is larger than ), causing their spatial low-pass filter to attenuate lower spatial frequencies. Thus, HC low-pass filters the signal while CT bandpass filters it, as shown in Fig. 1(c), with the same corner frequency. We can determine this peak spatial frequency, , by taking the derivative of CT’s system equation and equating it to zero. We find (3) To realize local automatic gain control in our model, we set HC activity proportional to intensity and use this activity to modulate CO to CT attenuation, , by changing cone-to-cone conductance. This modulation adapts cone sensitivity to different light intensities . Furthermore, to overcome the expansion in receptive-field size that this modulation caused in earlier designs, we complemented HC modulation of cone gap-junctions with HC modulation of cone leakage conductance, through shunting inhibition, making independent of luminance. We also compensated for the change in loop-gain with HC modulation of cone excitation, by keeping proportional to , thus fixing . ZAGHLOUL AND BOAHEN: OPTIC NERVE SIGNALS IN A NEUROMORPHIC CHIP I: OUTER AND INNER RETINA MODELS 659 We can determine how the CT activity produced by an edge depends on these parameters by assuming that the dominant contribution is from the peak and, hence, we insert the value into CT’s system equation of where we have set , assuming a proportionality constant represents the frequency component of unity for simplicity. of at . Hence, CT activity will be proportional to contrast if , which is determined by HC activity, is proportional to local intensity. We can determine the local HC activity and, hence, , ) and the by summing responses to the dc component (i.e., peak spatial frequency (i.e., ) where is the intensity level at which the CT attenuation, , is unity, and where represents the local average of input and as before. Therefore (4) From (4), we see that CT activity is primarily determined by , saturates when increasing signal flucspatial contrast, tuation causes this ratio to become large, and is entirely independent of absolute luminance. In the case when signal fluctuaincreases the level of tion, , is large, increasing the ratio , and have negcone saturation. In the limit where ligible effect on the system’s peak sensitivity at low contrasts ). (i.e., CT activity governed by (4) in our model exhibits behavior remarkably similar to the mammalian retina. Physiologists have found that cone responses as a function of intensity of light stimulation against a background, or contrast, are described by a simple equation where is the peak amplitude of the cone response produced by a given level of stimulating light intensity, . is the maximum response and is the background intensity. The responses . This adaptive reaches half of the maximum when behavior is preserved across five decades of background intensities. In our model, cone responses take on the same S-shaped . response with contrast when The complete outer retina CMOS circuit that implements local gain control and spatiotemporal bandpass filtering, while using HC modulation to maintain invariant spatial filtering and temporal stability, is shown in Fig. 2(a) (adapted from ) for , increasing two adjacent nodes. Photocurrents discharge CT activity, and excite the HC network through an nMOS transistor followed by a pMOS current mirror. HC activity, represented by , modulates this CT excitation and inhibits CT activity—by dumping this same current on to . Cone signals, , are electrically coupled to the six nearest neighbors Fig. 2. Outer retina circuit. (a) Circuit: A phototransistor draws current through an nMOS transistor whose source is tied to V and whose gate is tied to V . This transistor passes a current proportional to the product of CT and HC activity, thus modeling HC shunting inhibition. In addition, this current, mirrored through pMOS transistors, dumps charge on the HC node, V , modeling CT excitation of HC, and HC autofeedback. V sets the mean level of V , governing CT activity. (b) CT coupling is modulated by HC activity. A HC node [V in (a)] gates three of the six transistors coupling its CT node [V in (a)] to its nearest neighbors. through nMOS transistors whose gates are controlled locally [Fig. 2(b)]. These cone signals gate transistors that feed by currents into the BC circuit, such that increases in , which to, increase the BC activity. HC tracks the level we set signals also communicate with one another, through pMOS , transistors, but this coupling is modulated globally by since interplexiform cells that adjust HC coupling are not present in our chip . We rectify the cone signal using the circuit shown in Fig. 3 (described in ) to model complementary signaling by BCs. Briefly, CT activity is represented by a current, , that is inversely related to (from our description above). And we compare to a reference current, , that is in turn inversely related .1 Therefore, we define two new currents, to a reference bias, and , to simplify our description of the bipolar circuit. Mirroring these currents on to one another pre, which equals serves their differential signal, . in our circuit sets quiescent current levels—decreasing causes more current to flow through both channels—making equal to the rectification in our circuit incomplete. We set mean value of such that the difference is positive when light is brighter ( decreases) and negative when light is dimmer ( increases). We can see that as cone activity increases ( falls, translating to a rise in , and a corresponding fall in ), current is diverted through the ON channel, but this current level quickly saturates. On the other hand, as cone activity decreases ( rises, translating to a fall in , and a corresponding rise in ), current flows through the OFF channel and increases as the reciprocal of cone activity . Thus, our bipolar circuitry divides signals into ON and OFF channels, as expected, but the division is not symmetric because one side of our circuit is fixed to the refer. ence bias, III. INNER RETINA MODEL Our model of inner retina neurocircuitry, which realizes low-pass and high-pass temporal filtering, adapts temporal 1In practice, we cannot simply tie V to V because mean CT activity, V , is slightly higher than V , due to drain-voltage mismatch between nMOS and pMOS transistor-pairs in the outer retina circuit. 660 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 51, NO. 4, APRIL 2004 Fig. 3. BC circuit. (a) CT activity, V , drives a current, I , that is compared to a reference current, I , driven by a reference bias, V . By mirroring these currents on to one another, the difference between them is encoded by either I or I . V determines the level of residual dc signal present in I and I . (b) and I (top). When V V (i.e., I I ), residual dc currents are propotional The difference between I and I determines differential signalling in I to e . Directly plotting the difference between cone activity, I , and I yields the curves on bottom. Increases in cone activity cause ON currents to saturate while decreases in cone activity cause OFF currents to increase reciprocally. = = receives full-wave rectified input from ON and OFF BT and NA and, thus, rises above its baseline value of unity at both onset and offset. BT drives the sustained GC, GCs, which responds for the duration of the step, while the difference between BT and NA activity drives the transient GC, GCt, which decays to zero. From the block diagram in Fig. 5(a), we can derive the system level equations for NA and BT with the help of the Laplace transform (5) Fig. 4. Inner retina synaptic interactions. ON and OFF BCs relay cone signals to GCs, and excite NAs and WAs. NAs inhibit BCs, WAs, and transient GCs; their inhibition onto WAs is shunting. WAs modulate NA presynaptic inhibition and spread their signals laterally through gap junctions. BCs also excite local interneurons that inhibit complementary BCs and NAs. dynamics to input frequency and to contrast, and drives GC responses, is shown in Fig. 4. Briefly, BC inputs to the inner retina excite GCs, an electrically coupled network of wide-field amacrine cells (WAs), and narrow-field amacrine cells (NAs) that provide feedback inhibition on to the bipolar terminals (BTs) . WA, which receives excitation from both ON and OFF BT and inhibition from both ON and OFF NA, modulates presynaptic feedback inhibition from NA to BT. We use a large membrane capacitance to model the NA’s slow, sustained, response, which leads to a less sustained response at the BT through presynaptic inhibition . Push-pull inhibition is realized by a third set of amacrine cells in our model and is implemented by the ON-OFF BC circuit. These BT signals excite both sustained and transient GCs, but transient cells receive feedforward inhibition from NAs as well . Our model of the inner retina is described by the system diagram in Fig. 5(a). We derived this diagram from the synaptic interactions of Fig. 4 by modeling the NA as a low-pass filter. Responses of the different inner retina cell types in this model to a step input are shown in Fig. 5(b). BC activity is a low-pass filtered version of light input to the outer retina. Increase in BC causes an increase in BT and a much slower increase in NA. The difference between BT activity and gain-modulated NA activity determines WA activity, which in turn sets the gain of NA feedback inhibition on to BT and WA. Thus, after a unit step input, BT activity initially rises but NA inhibition, setting in later, attenuates this rise until BT activity is equaled by gain-modulated NA activity. WA represents our local measure of contrast and where is the gain of the excitation from BT to NA and where (6) where is the time constant of NA and represents WA activity, which determines feedback strength. From (5) and (6), we can see that BT high-pass filters and NA low-pass filters . This the BC signal; they have the same corner frequency, and, closed-loop time-constant, , depends on the loop gain therefore, on WA activity. For example, stimulating the inner retina with a high frequency would provide more BT excitation (high-pass response) than NA inhibition (low-pass response) to WA. WA activity and, hence, , would subsequently rise, reducing the closed-loop time-constant , until the corner frereaches a point where BT excitation equaled NA quency inhibition. This drop in , accompanied by a similar drop in , will also reduce overall sensitivity and advance the phase of the response. The system behavior governed by (5) and (6) is remarkably similar to the contrast gain control model proposed by Victor , which accounts for response compression in amplitude and in time with increasing contrast. Victor proposed a model for , is the inner retina whose high-pass filter’s time-constant, determined by a “neural measure of contrast,” . The governing equation is This model’s time-constant depends on contrast in much the same way that our model’s time constant depends on WA acis similar to our and where tivity (6), where Victor’s is represented by how much WA activity Victor’s ratio ZAGHLOUL AND BOAHEN: OPTIC NERVE SIGNALS IN A NEUROMORPHIC CHIP I: OUTER AND INNER RETINA MODELS 661 Fig. 5. Inner retina model. (a) System diagram: NA signals represent a low-pass filtered version of BT signals and provide negative feedback on to the BC. The WA network modulates the gain of NA feedback (X). WA receives full-wave rectified excitation from BT and full-wave rectified inhibition from modulated NA (double arrows). BT drives sustained GCs directly while the difference between BT and NA drives transient ganglion cells (GCt’s). (b) Numerical solution to inner retina model with a unit step input of 1 V. Traces show 1 s of ON cell responses for BC, BT, NA, GCt, and WA. Outer retina time constant, , is 96 ms; is 1 s. increases above its value at dc in our model. As this activity is sensitive to temporal contrast (see below), we propose that our WA cells are the anatomical substrate that computes Victor’s neural measure of contrast. is set by the local temporal In our model, the loop gain contrast. Because WA cells are coupled together through gap junctions, WA activity reflects inputs from BT and NA weighted across spatial locations. These pooled excitatory and inhibitory inputs should balance when the system is properly adapted (7) (8) where we define as the gap-junction current resulting from and are spatial differences in WA activity . and , computed by sumfull-wave rectified versions of ming ON and OFF signals. If all different phases are pooled spatially, these full-wave rectified signals will not fluctuate and will , will be proportional to amplitude. And if we ignore , yielding a temporal measure of contrast, simply be since it is the ratio of a temporal difference (high-pass signal, ) and a temporal average (low-pass signal, ). From (5), we ; hence, the loop gain is see that at dc, this ratio is equal to . unity and the dc gain GC responses in our inner retina model are derived from the BT and NA signals of (5). Specifically, BT signals directly excite both sustained and transient types of GCs, but transient cells receive feedforward NA inhibition as well. The system equations determining GCs’s and GCt’s responses, derived from (5), as a function of the input to the inner retina, , are (9) (10) When BT-to-NA excitation has unity gain , feedforward inhibition causes a purely high-pass (transient) response in GCt while GCs retains a low-pass (sustained) component. With a , the dc gain, , approaches 1/2 and BT’s small loop-gain, and GCs’s responses become all-pass. However, as the loop gain increases, decreases and these responses become high-pass. The change in with loop gain is matched in both BT and NA, and so taking the difference between these two signals always cancels the sustained component in GCt. Thus, GCt produces a purely high-pass response irrespective of the system’s loop gain. A. Adapting to Temporal Frequency Most physiological studies have characterized the retina’s response when stimulated with only one temporal frequency. We can adopt a similar approach to characterize our model’s GC responses, but to do so, we must determine how our system adapts to a single temporal frequency by deriving a mathematical expression for ’s dependence on contrast, , as well as input frequency, . As we stimulate our model with the same temporal frequency and contrast at all spatial locations, such that there is no difference between surround and center loop gain, we can set in (8), provided the spatial frequency is not too high. and Then we introduce the frequency dependencies of in order to determine the behavior of the system’s closed-loop . gain, We assume sinusoidal inputs, , with amplitude . We also assume that the frequencies we are interested in are less than the outer retina’s temporal cutoff, so that we can ignore and only outer retina filtering for the moment, and that have energy at these input frequencies, plus a dc component. Hence, we obtain the spectra (11) (12) where , , and are defined as above and is the input and are the residual activity (dc and offsets) frequency. in and , respectively. From our bipolar circuitry, we find . The source of NA residual activity, that is determined by , is explained below. and , and These currents only have energy at so, using Parseval’s relationship, the ratio of the magnitudes of these two currents is simply the square-root of the ratio of the energy contained in ’s two impulses over that in ’s. Setting to simplify the computation, we find (13) 662 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 51, NO. 4, APRIL 2004 thus, reduce the slope of its rise. Furthermore, although does not affect the temporal dynamics of the GCt’s response, we can see that mismatch in will introduce low-frequency responses in GCt. In summary, this analysis makes specific predictions , temporal frequency about how the open-loop time-constant , and contrast interact to determine the frequency response, when we stimulate with a single temporal frequency. Fig. 6. Change in loop gain with frequency. The system loop gain, wg , depends both on temporal frequency and on contrast. Plots of this relationship are shown on both a small (left) and large (right) frequency scale. Loop gain rises with temporal frequency, ! , and saturates at a point determined by the contrast level. For a given temporal frequency, higher contrasts generate a larger loop gain. gives the system loop-gain as a function of , , and , when is substituted for . Recall that , which determines . Numerical solutions means that the loop gain to this relationship are plotted in Fig. 6 for five different contrast of 1 s. levels, with a Finding approximations to (13) over different temporal frequency ranges, we obtain . , the system’s loop gain is In the dc case, when exceeds , , which sets the 1, as expected. As closed-loop time-constant, , tracks over the range . In this region, the inner retina effectively adapts to temporal frequency by matching its corner frequency to the input. The adaptation, however, only takes , the place over these intermediate frequencies—below , and above system’s corner frequency is floored at , it saturates at . We can use these loop gain expressions to determine the frequency response of our GC outputs. For temporal frequencies , we can make the approximation . below Ignoring the dc component, and invoking the relationship and , the GC frequency responses simplify to (14) B. Adapting to Contrast In addition to tracking temporal frequency, the inner retina also adapts to input contrast. We can see from Fig. 6 that loop-gain increases as we increase stimulus contrast and, , so as to hence, the system will adjust its corner frequency, increase it, causing a speed up in the GC response and rejecting a larger range of low frequencies. This contrast-dependence , saturates with arises because the point where loop gain, temporal frequency occurs at higher temporal frequencies for higher contrasts. This happens because NA’s response drops at frequencies above , making it below its dc offset impossible to track frequencies beyond this point. However, to explore how our model responds to contrast in natural scenes, which stimulate the retina with several temporal frequencies simultaneously, we need to understand how much each spectral component contributes to setting the loop gain, . For any individual frequency, WA activity will reflect what the adapted activity for that individual frequency ought to be only to a limited extent since the WA activity will reflect the weighted contributions of all the frequencies to loop gain. Thus, although sensitivity to all frequencies drops when stimulus contrast increases, low-frequency gains are attenuated more  since the system’s loop gain is pushed to a higher level by the higher frequencies than would normally be the case if low input frequencies solely determined system loop gain. We use the same approach as above to formalize this intuition, but we now integrate the energy over the entire spectrum acof temporal frequencies present to determine how and tivity affect loop gain, . We assume a flat spectrum, with amplitude proportional to contrast, that is assumed to be low-pass filtered by the outer retina. In this case, we define the contrast spectral density, , as contrast, or amplitude, per unit frequency. The relation between and the input signal contrast, , is given , since we divide the input spectrum’s energy, or as contrast squared, by the bandwidth we are interested in. In this and ’s spectra are given by case, (15) (16) , We can see from these equations that for input at dc to at . GCs’ response rises from . Hence, we Actually, it rises to at frequencies above expect that GCs’ response will be hardly affected by changes in since GCs’ response is practically flat across all . Finally, we expect GC’s response to remain independent of , since it does not appear in the equations. at On the other hand, the GCt response rises from at , following GCs’ behavior. Because dc to , GCt’s response is close to zero at dc. Reducing will shift the onset of its saturation range to higher frequencies and, (17) are defined as above. To facilitate where , , , , and our analysis, we approximate the outer retina as two low-pass filters, with an identical fixed time constant of , that sharply attenuate frequencies greater than . Sketches of and ’s spectra are shown in Fig. 7(a). To find the loop gain, we take the ratio of the magnitudes of and , computed using Parseval’s relationship. Simplifying ZAGHLOUL AND BOAHEN: OPTIC NERVE SIGNALS IN A NEUROMORPHIC CHIP I: OUTER AND INNER RETINA MODELS 663 Fig. 7. Effect of contrast on loop gain. (a) BT activity, i , is the sum of three components: a dc component that depends on residual BT activity, b , a low-pass component that equals d and cuts off at ! , and a high-pass component that rises as d ! , exceeds the low-pass component at ! 1= , and saturates at ! . The outer retina provides an absolute cutoff at ! . NA activity, i , is the sum of a dc component that depends on NA residual activity, n , and a low-pass component that equals dg and that cuts off at ! . Loop gain, wg , is determined by the ratio between the energy in i and the energy in i . (b) A numerical solution for loop gain as a function of contrast (c = d ! ) for three different levels of residual activity, b . As b increases, the curves shift to the right, implying that the contrast signal is not as strong. is 1.038 s and is 77 ms for this simulation. = p our analysis by setting and by treating the temporal cutoffs at and as infinitely steep, we find (18) (19) Recalling from (6) that is a function of loop gain , we can depends on input contrast. find a numerical solution to how This relationship is shown in Fig. 7(b) for three different values approaches 1 as contrast per unit frequency, of . Loop gain , approaches rises sublinearly with contrast over a range . The and saturates at a value determined by the ratio contrast that realizes this saturation is determined by the amount increases, the linear regime of residual BC activity . As shifts to higher contrasts. This implies that the determines the system’s contrast threshold—the higher , the more contrast we need to produce a given loop gain. This property, whereby the amount of residual activity determines the strength of the term from input contrast signal, is analogous to Victor’s (7). The gross behavior of this analytical solution is captured by the approximation (20) . Loop gain rises sublinearly with contrast and saturates when contrast exceeds some threshold, , that depends on . We can derive approximately what this threshold, , is by recognizing saturates when in the denominator of that the second term of (19). Assuming that in this case, given above as yields a the saturation value for . limiting contrast value of To better illustrate how varying levels of contrast affect the range of temporal frequencies over which adaptation is takes place, we can recognize that, below , loop gain quite well. Hence, our expressions approximated by for GC outputs (9) and (10) can be rewritten as (21) Fig. 8. Inner retina circuit. The inputs to the circuit, I and I , represent complementary inputs to the rectifying BC circuit. We make five copies at the output of this circuit (only three are shown in the figure). One copy drives the ON-OFF low-pass filter to excite the NA on one side of the circuit while inhibiting NA on the other side. A second copy is used to excite WAs, which modulates NA feedback inhibition on to BT. The remaining three copies feed ON and OFF versions of transient and sustained GCs (OnT, OnS, OffT, and OffS). (22) Frequencies below are attenuated by while frequencies above are not. Thus, the amount of attenuation, as well as the range of frequencies over which this attenuation takes place, gets larger with increasing contrast. From our analysis, we see that both temporal frequency and contrast are interrelated in their effect on the dynamics governing our inner retina model. Furthermore, because of the relationship between temporal frequency and contrast, changes in one of these parameters cause changes in the model’s adaptation to the other. Increasing either one of these parameters causes the time constants of our model both to speed up, which affects how our inner retina processes natural images. This adaptation to contrast and temporal frequency makes retinal coding more efficient, from an information theoretic perspective (see Discussion). C. Circuit Implementation The complete inner retina circuit that realizes the synaptic interactions of Fig. 4 is shown in Fig. 8 (described in detail in and ). Briefly, differential signals from BC circuitry, 664 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 51, NO. 4, APRIL 2004 , drive either half of the circuit. Excitatory BT-to-NA current excite the NA node on one side of our circuit while also inhibiting the NA node on the complementary side, thus realizing push-pull inhibition. These inputs are divided by the sum of ON and OFF NA activity to compensate for the exponential voltage-current relationship in the subthreshold regime. NA provides feedback inhibition on to BT, modulated by WA activity. Convergence of ON and OFF signals on to WA implements full-wave rectification for both BT excitation and NA inhibition of WA. WA nodes are coupled to one another through an nMOS . A copy of NA current inhibits diffusion network gated by the transient GC. In fact, our circuit generates two copies of transient signals so that we can pool transient GC inputs over larger areas . Because we have control over both WA coupling and BT-to-NA gain , we can explore how changing these parameters changes GC responses. As WA modulation determines the dynamics of GC responses, we expect the extent , to of spatial coupling in the WA network, controlled by affect circuit dynamics. In our model, WA activity reflects inputs weighted across spatial locations, and is influenced by differences in center and surround WA activity. Relative contributions from different loop gains at different spatial locations are determined by the resistance of the WA network, in (8). Here, is the difference between with WA activity in a surround location, , and a center location, , and is the resistance coupling these two locations. Thus, depending on the strength of WA coupling, if WA activity in the surround is larger than that in the center, we expect the loop gain in the center to increase, whereas if the opposite is true, we expect loop gain in the center to decrease. BT-to-NA gain is determined by the relationship between , , and . sets the source voltages for the NA subcircuit such that WA activity can be represented by voltage . A bias voltage, , determines leakage deviations below current, , from NA’s large membrane capacitor and, hence, the time constant for both common-mode and differential signals. And the gain of BT-to-NA excitation is determined by a , applied to the source of the current mirror’s bias voltage, should be set equal to for input transistor. Ideally, a gain of one . If , then the gain, , is greater than one, thus, WA activity should be lower and GCt responses , then is less should be inhibited. However, if than one, causing the opposite effects . Finally, another bias in the model’s voltage, , sets quiescent NA activity (i.e., equation) by exciting NA on both sides of the circuit with a current that is divided by ON and OFF NA activity (not shown). This ensures that ON and OFF NA activity are limited by a geometric does in the BT circuit). mean constraint (just like IV. DISCUSSION In this paper, we have presented our model and analyzed its behavior. In order to construct our model, and in order to facilitate its analysis, we made several simplifications. We address these assumptions and present our justifications here. While our analysis of the outer retina demonstrates how our model filters signals in space, capacitances associated with the cone and HC networks create time constants that produce temporal filtering that is inseparable from spatial filtering. Details of our outer retina’s temporal properties and their effect on spatial filtering are discussed elsewhere –. Briefly, CT responses in our model change from bandpass to low-pass in space as temporal frequency increases. They also change from bandpass to low-pass in time as spatial frequency increases. This interplay between spatial and temporal filtering creates a CT response that peaks at a single temporal frequency for low spatial frequencies and at a single spatial frequency for low temporal frequencies. HCs modulate synaptic interactions to realize luminance adaptation in our outer retina model, but in the mammalian retina this role is most likely played by dopaminergic amacrine cells  and light sensing GCs . Although temporal filtering in the outer retina is not addressed in this paper, we do use a simplified version of the outer retina’s temporal filter for our analysis of contrast adaptation in the inner retina. This analysis, presented in Section III, models the outer retina as two low-pass filters with the same time constant, . We felt this simplification was valid because our inner retina analysis is limited to high spatial frequencies where WA averaging across space occurs. At these high spatial frequencies, spatiotemporal filtering in the outer retina is indeed low-pass in time –. We examined responses in this simpler condition to determine how the inner retina adapts to temporal frequencies and contrast. The same adaptation process still takes place when a more complex spatiotemporally filtered signal is presented by the outer retina and, hence, the simple picture we have painted may provide useful intuition. Our inner retina model is based upon the functional architecture and physiological behavior of the mammalian retina. All the synaptic interactions we included have been identified anatomically, as well as the complementary ON and OFF channels. Complementary signaling starts at cone-to-bipolar synapses, by using a sign-reversing synapse in one channel and half-wave rectification in both . Push-pull interactions maintain complementary signaling in the inner retina through reciprocal inhibition between ON and OFF channels, via vertically diffuse amacrine cells that span the ON and OFF laminae . Serial inhibition  may also play a vital role in these interactions. While the outer retina adapts its sensitivity to light intensity, the inner retina adapts its low-pass and high-pass temporal filters to contrast and frequency. Encoding signals found in natural scenes optimally requires the retina’s bandpass filters to peak at the spatial and temporal frequency where input signal power drops to the noise floor , . Therefore, as stimuli with different spectra are presented to the retina, optimal coding requires this filter’s peak frequency to move accordingly. Thus, the retina must adapt to spatial and temporal frequency and to luminance and contrast to continue to convey information efficiently to higher cortical structures. As stimulus power increases—as in the case of increased contrast—optimal filtering demands that the peak of this bandpass filter moves to higher frequencies, with a corresponding drop in low-frequency sensitivity. Our inner retina model’s temporal filter realizes this adaptation to contrast—GC responses compress in time and amplitude when driven by steps of increasing contrast —by adjusting ZAGHLOUL AND BOAHEN: OPTIC NERVE SIGNALS IN A NEUROMORPHIC CHIP I: OUTER AND INNER RETINA MODELS its time constant , . Our model proposes that this adaptation is implemented through WA modulation of NA feedback (pre-synaptic inhibition). Thus, we offer an anatomical substrate through which earlier dynamic models can be realized. A likely biological candidate for this WA is the A19 amacrine cell , which lies at the border between ON and OFF laminae, has thick dendrites, a large axodendritic field, and is electrically coupled to other A19 amacrine cells through gap junctions. Our inner retina circuit design also corrects flaws in the design in , which failed to produce temporal adaptation, and adds GC level synaptic interactions and convergence. Our retina model’s output is encoded as spike trains in four types of GC outputs. We sought to model parasol (also called Y in cat) and midget (also called X in cat) GCs, which respond in a transient and in a sustained manner, respectively, at stimulus onset or offset. Both types of GCs receive synaptic input from BCs and amacrine cells, although parasol cells receive more amacrine synapses, and, presumably, more feedforward inhibition , . Hence, we included feedforward inhibition in our transient cells, but not in sustained cells. Furthermore, parasol cells also sample the visual scene nine times more sparsely than midget cells, and have proportionately larger receptive fields . Thus, we tiled our transient cells less densely, and included convergence of bipolar signals on to these cells (see ). Ninety percent of the total primate GC population is made up of ON and OFF midget and parasol cells  and so we concentrated our modeling efforts on these four cell types. V. CONCLUSION By morphing the retina’s functional architecture into a VLSI circuit, we were able to create a circuit that expresses some of the defining features of visual processing in the mammalian retina. In our outer retina model, interaction between an excitatory cone network, which has relatively small space and time constants, and an inhibitory HC network, which has larger space and time constants, creates a bandpass spatiotemporal response. This model adapts to input luminance through HC modulation of cone-coupling and cone excitation, and through autofeedback, to produce a contrast signal at the CT. From our analysis, we see that CT activity is determined by input contrast, and independent of mean luminance. Hence, our model’s CT’s S-shaped behavior conveys a contrast signal to subsequent retina circuitry that does not change with light intensity, similar to the behavior exhibited by the mammalian retina. BCs in our model rectify these signals into ON and OFF channels to replicate the retina’s complementary signaling. Our design was based on push-pull signaling in the mammalian retina that divides signals into complementary pathways to ensure efficient coding. Furthermore, by morphing the inner retina’s functional architecture into a VLSI circuit, we were able to extend previous models of visual processing in the mammalian retina. In our inner retina model, modulation of NA presynaptic inhibition by WAs realizes contrast gain control and time-constant adaptation. WAs serve as an anatomical substrate for computing a neural measure of contrast in our model. We use this measure of contrast in the same way that the retina is hypothesized to—modulation of feedback inhibition changes the system’s closed-loop 665 time constant. This adaptation allows our model to optimally encode signals by adjusting its temporal filter based on input frequency and contrast. Thus, in response to natural scenes, our model measures all input frequencies present and computes how the system should best filter these signals. A number of parameters in our model and circuit determine the system’s behavior. Manual control over HC coupling in our outer retina model allows us to explore how changing space constants in this network affects GC output. By adjusting the NA leakage current, we also have control over the system’s open-loop time constant. Changing this time constant affects the circuits temporal response and changes the temporal frequency above which closed-loop gain saturates. And by adjusting the BT to NA gain, we have control of residual transient cell activity. Finally, by adjusting coupling strength in the WA network, we can explore the effect of spatial frequency on how loop gain is computed over space. We explore some of these parameters in a companion paper , where test results from a fabricated chip are presented. By reproducing some of the synaptic interactions found in the mammalian retina, we were able to derive models for the outer and inner retina that spatiotemporally filter signals, adapt to luminance, and exhibit contrast gain control. Our procedure has been to construct a neural model based on known anatomy in order to elicit some of the known behavior found in the physiology. While our model assigns novel roles to circuit elements that can account for this behavior, we still need to validate these conjectures by comparing our model’s behavior with that of the mammalian retina. This detailed comparison will be presented in a forthcoming paper. REFERENCES  J. Atick and N. Redlich, “What does the retina know about natural scenes,” Neural Computation, vol. 4, no. 2, pp. 196–210, 1992.  D. M. Berson, F. A. Dunn, and M. Takao, “Phototransduction by retinal ganglion cells that set the circadian clock,” Science, vol. 295, no. 5557, pp. 1070–3, 2002.  K. Boahen, “A retinomorphic chip with parallel pathways: Encoding increasing, on, decreasing, and off visual signals,” J. Analog Integr. Circuits Signal Processing, vol. 30, pp. 121–135, 2002.  K. A. Boahen, “The retinomorphic approach: Pixel-parallel adaptive amplification, filtering, and quantization,” in Anaglog Integr. Circuits Signal Processing, vol. 13, 1997, pp. 53–68.  K. A. Boahen and A. Andreou, “A contrast-sensitive retina with reciprocal synapses,” in Advances in Neural Information Processing 4, J. E. Moody, Ed. San Mateo, CA: MorganKaufmann, 1991, vol. 4, pp. 764–772.  W. Craelius, “The bionic man: Restoring mobility,” Science, vol. 295, no. 5557, pp. 1018–21, 2002.  L. J. Croner and E. Kaplan, “Receptive fields of p and m ganglion cells across the primate retina,” Vis. Res., vol. 35, no. 1, pp. 7–24, 1995.  J. B. Demb, K. A. Zaghloul, L. Haarsma, and P. Sterling, “Bipolar cells contribute to nonlinear spatial summation in the brisk-transient (y ) ganglion cell in mammalian retina,” J. Neurosci., vol. 21, pp. 7447–7454, 2001.  R. Douglas, M. Mahowald, and C. Mead, “Neuromorphic analogue vlsi,” Annu Rev Neurosci, vol. 18, pp. 255–281, 1995.  J. E. Dowling and B. B. Boycott, “Organization of the primate retina: Electron microscopy,” in Proc. Roy. Soc. Lond. B, vol. 166, 1966, pp. 80–111.  M. A. Freed and P. Sterling, “The on-alpha ganglion cell of the cat retina and its presynaptic cell types,” J. Neurosci., vol. 8, no. 7, pp. 2303–20, 1988.  R. J. Jensen and N. W. Daw, “Effects of dopamine and and its agonists on the antagonists on the receptive field properties of ganglion cells in the rabbit retina,” Neuroscience, vol. 17, no. 3, pp. 837–855, 1986. 666 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 51, NO. 4, APRIL 2004  M. Kamermans, I. Fahrenfort, K. Schultz, U. Janssen-Bienhold, T. Sjoerdsma, and R. Weiler, “Hemichannel-mediated inhibition in the outer retina,” Science, vol. 292, no. 5519, pp. 1178–80, 2001.  M. Kamermans and F. Werblin, “Gaba-mediated positive autofeedback loop controls horizontal cell kinetics in tiger salamander retina,” J. Neurosci., vol. 12, no. 7, pp. 2451–63, 1992.  H. Kolb, N. Cuenca, H. H. Wang, and L. Dekorver, “The synaptic organization of the dopaminergic amacrine cell in the cat retina,” J. Neurocytol., vol. 19, pp. 343–366, 1990.  H. Kolb and R. Nelson, “Functional neurocircuitry of amacrine cells in the cat retina,” in Neurocircuitry of the Retina: A Cajal Memorial, A. Gallego and P. Gouras, Eds. New York: Elsevier, 1985. , “Off-alpha and off-beta ganglion cells in cat retina: II. neural  circuitry as revealed by electron microscopy of hrp stains,” J. Comp. Neurol., vol. 329, no. 1, pp. 85–110, 1993.  G. Maguire and P. Lukasiewicz, “Amacrine cell interactions underlying the response to change in tiger salamander retina,” J. Neurosci., vol. 9, pp. 726–35, 1989.  C. Mead, Analog VLSI and Neural Systems. Reading, MA: AddisonWesley, 1989.  R. A. Normann and I. Perlman, “The effect of background illumination on the photoresponses of rod and green cones,” J. Physiol., vol. 286, pp. 491–507, 1979.  J. P. Rauschecker and R. V. Shannon, “Sending sound to the brain,” Science, vol. 295, no. 5557, pp. 1025–29, 2002.  R. W. Rodieck, “The primate retina,” Comp. Primate Biol., vol. 4, pp. 203–278, 1988.  B. Roska and F. Werblin, “Vertical interactions across ten parallel, stacked representations in the mammalian retina,” Nature, vol. 410, pp. 583–587, 2001.  R. Shapley and J. D. Victor, “The contrast gain control of the cat retina,” Vis. Res., vol. 19, pp. 431–434, 1979.  P. Sterling, Retina, and G. M. Shepherd, Eds., The Synaptic Organization of the Brain, fourth ed. New York: Oxford Univ. Press, 1998.  J. B. Troy and C. Enroth-Cugell, “ and ganglion cells inform the cat’s brain about contrast in the retinal image,” Exp. Brain Res., vol. 93, pp. 383–390, 1993.  J. H. van Hateren, “A theory of maximizing sensory information,” Biological Cybern., vol. 68, pp. 23–29, 1992.  J. D. Victor, “The dynamics of cat retinal cell centre,” J. Physiol., vol. 386, pp. 219–246, 1987.  K. A. Zaghloul and K. Boahen, “An on-off log-domain filter circuit,” IEEE Trans. Circuits Syst., 2004, to be published.  , “Optic nerve signals in a neuromorphic chip II: Testing and results,” IEEE Trans Biomed. Eng., vol. 51, pp. 667–675, Apr. 2004. X y x  , “A silicon model of the mammalian retina,” Neuron, 2004, submitted for publication.  E. Zrenner, “Will retinal implants restore vision,” Science, vol. 295, no. 5557, pp. 1022–25, 2002. Kareem A. Zaghloul (S’00–M’03) received the B.S. degree from the Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, in the in 1995, where he made Tau Beta Kappa and Eta Kappa Nu. He recently completed a combined M.D./Ph.D. program at the University of Pennsylvania, Philadelphia. The Ph.D. degree was awarded in the Department of Neuroscience, where he worked on understanding information processing in the mammalian retina with K. A. Boahen. His work was supported by a Vision Training Grant from the National Institutes of Health and a Ben Franklin Fellowship from the University of Pennsylvania School of Medicine. He is currently a Resident Physician in the Department of Neurosurgery at the University of Pennsylvania. Kwabena A. Boahen received the B.S. and M.S.E. degrees in electrical and computer engineering from the Johns Hopkins University, Baltimore MD, in the concurrent masters-bachelors program, both in 1989, where he made Tau Beta Kappa. He received the Ph.D. degree in computation and neural systems from the California Institute of Technology, Pasadena, in 1997, where he held a Sloan Fellowship for Theoretical Neurobiology. He is an Associate Professor in the Bioengineering Department, University of Pennsylvania, Philadelphia, where he holds a secondary appointment in the electrical engineering. He was awarded a Packard Fellowship in 1999. His current research interests include mixed-mode multichip VLSI models of biological sensory and perceptual systems, and their epigenetic develoipment, and asynchronous digital interfaces for interchip connectivity. Dr. Boahen received a National Science Foundation (NSF) CAREER Award in 2001 and an Office of Naval Research (ONR) YIP Award in 2002.
* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project