Sense Element Engagement Theory Explains How Neural Networks Produce Cortical Prosthetic Vision

Demonstrating that an understanding of how neural networks produce a specific quality of experience has been achieved would provide a
foundation for new research programs and neurotechnologies. The phenomena that
comprise cortical prosthetic vision have two desirable properties for the
pursuit of this goal: 1) Models of the subjective qualities of cortical
prosthetic vision can be constructed; and 2) These models can be related in a
natural way to models of the objective aspects of cortical prosthetic vision. Sense
element engagement theory portrays the qualities of cortical prosthetic vision
together with coordinated objective neural phenomena as constituting sensible
spatiotemporal patterns that are produced by neural interactions. Small-scale neural
network simulations are used to illustrate how these patterns are thought to
arise. It is proposed that simulations and an electronic neural network (ENN)
should be employed in devising tests of the theory. Large-scale simulations can
provide estimates of parameter values that are required to construct an ENN.
The ENN will be used to develop a prosthetic device that is predicted by the
theory to produce visual forms in a novel fashion. According to the theory, confirmation
of this prediction would also provide evidence that this ENN is a sentient
device.


I. INTRODUCTION
HE theory that is presented here asserts that the phenomena of cortical prosthetic vision are self-organized patterns produced by neural interactions. I focus on cortical prosthetic vision (CPV) because understanding how neural interactions produce any quality of experience would be a scientific milestone and set the stage for technological innovations [1] and because the subjective experiences that arise in CPV and their neural basis are amenable to modeling.
Sense element engagement theory is made possible by the perspective that is adopted in conceptualizing how a quality of experience can arise from the physical behavior of networks of interacting neurons. Providing such a conceptualization has been termed the "hard problem" [2], [3]. Although previous work has revealed suggestive relationships between patterns of This paragraph of the first footnote will contain the date on which you submitted your paper for review. It will also contain support information, including sponsor and financial support acknowledgment. For example, "This work was supported in part by the U.S. Department of Commerce under Grant BS123456." activity in neural network simulations and qualitative and quantitative aspects of specific visual experiences (e.g., [1], [4], [5], [6]), the physical implications of the relationships are not understood. The perspective on the hard problem that is adopted here allows us to comprehend the experiences reported in CPV and certain coordinated neural interactions as jointly constituting sensible spatiotemporal patterns and to predict that altering a specific aspect of neural interactions will produce a specific change in a visual quality of these patterns.
The remainder of the paper is organized as follows. Section II provides an overview of CPV with an emphasis on research which suggests how CPV can be modeled. Section III provides a description of the perspective that is taken with respect to the hard problem and illustrates the novel and scientifically useful view of CPV that this perspective offers. Sense element engagement theory [7] is presented in section IV and is illustrated using neural network simulations. Section V presents an argument for employing large-scale simulations to develop an ENN which will provide a testbed to be used in development of a prosthesis that is predicted to produce visual forms in a novel way. Section VI shows that sections II through V provide a process that can be applied to other qualities of experience.

II. OVERVIEW OF CORTICAL PROSTHETIC VISION
CPV systems can potentially provide rudimentary vision for many blind individuals [8]. In these systems, a microprocessor uses input from a video camera mounted in eyeglasses to electrically stimulate neurons in primary visual cortex (V1), producing phosphenes (bright spots) from parts of the image. This technology is employed by the Orion Cortical Visual Prosthesis System [9] and by similar systems that are in various phases of development and/or clinical testing [10], [11], [12].
It has been known since the early twentieth century that stimulation of a spatially localized population of neurons in V1 results in the experience of a phosphene (see reviews in [13], [14]). However, the feasibility of using cortical stimulation for prosthetic vision was not explored until the publication of a dramatic demonstration in 1958 [15]. From that time until the Fig. 1. Simulated phosphene representations of the uppercase letter "X". (a) is a discrete phosphene representation based on results in [26], (b) is a simulation of a representation based on an alteration of the visual geometry in which phosphenes appear, and (c) is a representation that is based on an alternative alteration of the visual geometry. The concepts underlying (b) and (c) are described in section V. present, direct cortical stimulation during neurosurgery and stimulation controlled by video images have been used to produce multiple distinct phosphenes each of which is experienced as occupying a continuous region in visual space [13], [16], [17], [18], [19], [20], [21], [22]. An implant recipient can point to the perceived locations of phosphenes [23] and can describe their brightness, size and shape as though they were physical objects in the environment [13].
An increase in phosphene size with visual eccentricity [13], [24] is attributed to the well-known V1 cortical magnification factor [21]. V1 is organized functionally into cortical columns containing circuits responsible for identifying the opponent-color wavelength components of light, orientation, movement, and spatial extent of contrast, and binocular disparities for input that originates in specific regions of the retinas (e.g., [25]). The magnification factor describes the observation that a very small area in the central portion of each retina is represented by neurons in a cortical column of V1, and as retinal eccentricity increases an increasingly large area is represented by a cortical column. Consistent with cortical magnification, stimulating neurons within a column that represents a central portion of the retina yields a much smaller phosphene than does stimulating neurons within a column that represents a more peripheral portion of the retina [13], [24].
It has been found in a group of human volunteers that the size of phosphenes also increases with stimulating current up to a value at which the effect saturates [26]. Based on experiments conducted in macaque monkeys [27], [28], it has been estimated that 1.9 (the lowest current delivered through a microelectrode that was used for the induction of phosphenes) activates a volume of 6.2 × 10 −4 3 of tissue containing 75 neurons [24]. The same authors also estimated that currents less than 100 would be necessary in order to activate subregions of a column using a microelectrode that penetrates the surface of the cortex. The currents used with surface electrode implants are typically much larger and likely stimulate a large proportion of the neurons in a cortical column. For example, [26] reported a mean threshold of 0.81 for evoking a phosphene in their group of human volunteers. Fig. 1(a) depicts a simulated phosphene representation of the uppercase letter "X" based on results reported by [26]. The phosphenes have been rotated about the coordinates of the center of the letter in order to minimize the sum of the distances of each phosphene from a corresponding pixel representation of the letter. When appropriately sized and viewed from a distance of 54 , the letter is centered at (−7.5°, 8.5°), from the direction of gaze in degrees of visual angle. As illustrated by Fig. 1(a), implant recipients perceive phosphenes within regions of visual space that are determined by the positions of stimulating electrodes. Electrodes are implanted some distance away from the cortical columns that represent the central portion of each retina so that each phosphene is located some distance from the direction of gaze. An attempt to "look at" a phosphene by moving the eyes causes its apparent position in the environment to change so that it retains a fixed visualspatial relationship to the direction in which the eyes are pointing [20], [22]. Fig. 1(b) suggests that visual fusion of appropriate pairs of phosphenes that produce a single unified visual image for each letter might improve cortical prosthetic vision. With one exception, perception of continuous visual forms has never been reported. When current is simultaneously delivered through multiple electrodes, an implant recipient reports seeing multiple discrete phosphenes and must infer what object in the environment they represent [13], [22], [29]. Recently [17] reported the perception of continuous visual forms in a group of sighted and blind volunteers. They used a technique known as current steering [30] to "trace" the pattern of each letter by activating electrodes sequentially and found dramatic increases in the ability of volunteers to identify shapes correctly. It is suggested in [17] that this technique may activate visual motion circuits that, in turn, result in form perception. Although this is a plausible suggestion, it does not address the question of how the experience of either discrete or unified phosphenes or of the visual space which phosphenes populate arises from activity in neural networks. Fig. 1(c) is based on a prediction that follows from the sense element engagement theory and is discussed in section V.
The results of CPV research suggest that the visual geometry in which phosphenes appear can be modeled using an idealized relationship between coordinates in V1 and visual coordinates. Cartesian coordinates ( , ) of a flattened geometry of the V1 surface can be obtained as functions of retinal image eccentricity and azimuth coordinates (e.g., [31], Ch. 2), (1) Equations (1) are derived in [31] by assuming that ( ) is a function of eccentricity only and that ( , ) is proportional to , and by using results from the macaque monkey [32] to obtain the relationship ( ) = ( 0 + ⁄ ), where ( ) is the magnification factor in mm/degree. The quantities 0 and are estimated from data. The inverses of (1) can then be used to approximate visual geometry coordinates from V1 coordinates as in which ̂ and ̂ denote values of a visual geometry eccentricity coordinate and a visual azimuth coordinate, respectively and 0 and are order parameters of a visual geometry pattern. Fig. 2 illustrates an idealized flattened geometry of V1 with columns depicted as circular disks and a corresponding visual geometry that results from applying (2) with = 12 and 0 = 1°, values that fit data obtained from the macaque monkey [32]. Overlapping regions in the visual geometry that correspond to columns are constructed by transforming coordinates ( , ) of the center of V1 disk to values (̂,̂) of idealized visual geometry coordinates, and assigning the value (0.15 + 0.09 ̂) 1.2 to the radius of visual geometric region ̂. Finally, (̂,̂) values are converted to cartesian coordinates (̂°,̂°) that specify the center of each visual region in degrees of visual angle.
In order to model phosphenes we require an equation comparable to (2) that characterizes the relationship of lightness values to activity in lightness channel neurons. The typically achromatic nature of phosphenes (e.g., [13], [14]) justifies the assumption that stimulation activates both Blue-Yellow and Red-Green opponents approximately equally. This assumption seems reasonable given the likelihood that stimulation current produces additional action potentials at various sites on the axons of neurons across cell types and cortex layers [33], [34]. Furthermore, the absence of complexities such as a dependency of lightness on context in CPV suggests that phosphenes can be modeled using a functional relationship between a measure of neural activity in the lightness channels of each V1 column and visual lightness values. If ( , , ) is the action potential It also seems likely that a range of action potential frequencies and of lightness values are produced within each column, and therefore (3) will be used to construct a lightness interval Λ (̂°,̂°, ) = [ (̂(̂°,̂°, )) , (̂(̂°,̂°, ))] on each ̂ from different values of in lightness channel neurons. The resulting network-wide lightness interval distribution pattern is described by order parameters 0 and 0 .

III. A PRAGMATIC VIEW OF THE HARD PROBLEM
It is possible to deal with the hard problem in a practical and useful way using ideas put forward by [35], [36], and [37]. In chapter 12 of "The Nature of the Physical World," Eddington [35] notes that our measurements provide all that we can know of what are necessarily enigmatic physical phenomena, that is, phenomena the intrinsic nature of which we cannot know. He states that this situation compels us to attach our measurements to a background that is consistent with the existence of those phenomena. For example, we implicitly attach measurements of the elements of patterns and structures that involve quantities such as displacement or duration to space and time backgrounds that are consistent with the existence of those measurements. When Eddington turns to the mental aspects of doing natural science as comprising additional natural phenomena requiring explanation, he states that it would be "silly" to attach thought to something of a "concrete" nature that is inconsistent with the qualities of experience and then to wonder where those qualities come from ( [35], pp. 257-260).
In the year following Chalmers' [3] initial portrayal of the hard problem, Hut and Shepard [36] described a perspective which led them to suggest a sense or background, on a par with space ( ) and time ( ) backgrounds, that provides the "condition of possibility for experience" (see also [37].) The sense background seems to be very well aligned with Eddington's request that we accept the significance of the background to which we attach our measurements, "namely that it has a nature capable of manifesting itself as mental activity" ( [35], p. 260).
The sense element engagement theory of CPV assumes the existence of a sense background. Phenomena that characterize the objective spatiotemporal ( ) aspects and the subjective sensible-temporal ( ) aspects of CPV are viewed as projections of macroscopic self-organized patterns in space, sense and time (i.e., patterns) that arise from neural interactions which meet specific criteria. What the theory adds to accounts of behavioral and neural self-organized patterns (e.g., [38], [39]) is the proposal that interactions take place in space, time, and sense.
From this perspective the issue of whether the qualities of CPV experience influence neural activities, a difficulty for a viewpoint that is referred to as "the standard scientific approach" in [36] does not arise. According to the sense element engagement theory, the observed coordination of visual experience with objective neural quantities should be described by mappings → in analogy with the manner in which the observed coordination of spatial location with measured time during motion is described in the form of mappings → . In classical mechanics, mappings → are found by integrating a mathematical statement of Newton's second law. In the case of cortical prosthetic vision, the theory proposes a process by which mappings → result from neural interactions. In neither case is a causal relationship from the domain to the range of the mapping implied.
In keeping with the perspective that is advanced in [36], interactions are viewed as partaking of the sense background as they partake of the time and space backgrounds. Continuing with the familiar Newtonian example, there are no instantaneous changes in spatial position, and the mappings → that reflect this empirical fact reveal how physical interactions necessarily partake of time. The sense element engagement theory rests on an assumption that certain interactions necessarily engage specific elements of patterns in in a manner that is reflected by common neural networkwide mappings → .
This assumption has a number of implications. The subjective qualities of visual geometry such as visual distance, direction, and size and the qualities of the lightness interval distribution such as differences in lightness between visual regions are taken to be properties of the elements of a projection of the pattern on just as correlated objective quantities such as interaction strengths are properties of the elements of a projection of the pattern on . The theory asserts that a specific neural interaction engages an element of a subjective visual geometry pattern or an element of a lightness interval distribution pattern as we might assert that the interaction occupies or occurs within some volume of a spatial pattern.
A description of the theory follows. Until sufficient data become available the theory must remain agnostic with respect to whether the visual geometry and lightness interval distribution result from interactions between populations of neurons or between single neurons. Because the latter case is much simpler to implement in computer simulations and an ENN, synaptic interactions between single neurons will be referred to throughout the remainder of the paper.

A. Synaptic Interactions in Cortical Prosthetic Vision
Sense element engagement theory posits a geometric (G) system of synaptic interactions in a V1 network that is responsible for a CPV geometry pattern in . When a stable network-wide visual geometry pattern is present, each active G system interaction engages an element of the pattern that is modeled as a set of vectors of the same magnitude that begin in a visual geometric region and terminate in the same or in a different visual geometric region, the presynaptic neuron engages the region in which these vectors begin, and the postsynaptic neuron engages the region in which the vectors terminate. The strength of G system synaptic interactions is a mathematical function of the magnitude of the geometric vectors. Interacting neurons in the same cortical column engage almost identical regions, neurons in adjacent columns engage overlapping regions, and regions grow with eccentricity in a manner that is consistent with the magnification factor of V1.
There are multiple G system synapses from each presynaptic neuron to each postsynaptic neuron and the interactions produced by these synapses vary in strength. When a stable pattern is present the collection of interactions from a given presynaptic neuron to a given postsynaptic neuron engages a collection of sets of geometric vectors, the vectors in each set having a different magnitude. The sets of vectors are such that each point in the presynaptic visual geometric region is related spatially to one or more points in the postsynaptic visual geometric region, and by symmetry each point in the postsynaptic visual geometric region is related spatially to one or more points in the presynaptic visual geometric region. The entire collection of geometric vectors between a presynaptic region and a postsynaptic region models the fuzzy (i.e., not point-to-point) qualities of visual distance and direction within the visual geometry.
The theory also includes G system synapses between a head representation network that is probably located in primary somatosensory cortex (S1) and the V1 network. Synaptic connections between S1 and V1 have been identified [40], and there is evidence for cross-modal influences [41]. G system synapses between S1 and V1 account for the subjective orientation of the visual geometry with respect to geometric regions of the head. Modulation of G system synaptic strengths accounts for changes in synaptic strengths that accord with changes in visual distance from the head to phosphenes on the visual geometry as would be expected to result from changes in vergence (as noted by [21]) and with correlated changes in the sizes of regions that are occupied by phosphenes.
A lightness (L) system of synapses between the same neurons that interact via G system synapses produces a lightness interval distribution on the geometry in SXT. When a stable macroscopic lightness interval distribution is present each active L system synaptic interaction engages a set of scalar lightness differences of the same magnitude. Each of the interacting neurons engages a lightness interval. The strength of interaction is a function of the absolute value of the difference between pairs of values in the presynaptic and postsynaptic intervals. Modulation of L system synaptic strengths accounts for changes in synaptic strengths that accord with changes in the distribution of lightness intervals on the visual geometry.
There are multiple L system synapses from each presynaptic neuron to each postsynaptic neuron and the interactions produced by these synapses vary in strength. When a stable lightness interval distribution is present the collection of interactions from a given presynaptic neuron to a given postsynaptic neuron engages a collection of elements of the lightness interval distribution pattern that is modeled as a collection of sets of scalar differences, the differences in each set having a different magnitude. The sets of differences are such that each value in the presynaptic scalar interval is related by a difference to one or more values in the postsynaptic scalar interval, and by symmetry each value in the postsynaptic scalar interval is also related to one or more values in the presynaptic scalar interval. The entire collection of scalar differences between a presynaptic interval and a postsynaptic interval models the fuzzy quality of difference in lightness between geometric regions.
Although details of how pattern identification or recognition and behavioral report occur are beyond the scope of the present theory, the fact that implant recipients are aware of what a lightness interval distribution pattern represents (e.g., the identity of an uppercase letter) should be noted. In particular, this phenomenon implies that output of the V1 network results in interactions within neural networks beyond V1 that generate patterns the projections of which constitute awareness of what a lightness interval distribution pattern represents. Furthermore, this can be communicated behaviorally.

B. Self-Organization of the Visual Geometry and the Lightness Interval Distribution Patterns
The operational approach to understanding pattern formation in complex biological systems described in [38] will be utilized to explain how CPV patterns are thought to arise. Proposals regarding self-organization of these patterns will be illustrated using small-scale neural network simulations.
Although the visual geometry is characterized by two order parameters, (2) shows that very small positive values of 0 result in small values of eccentricity ̂ for which the visual geometry itself vanishes. Variations in 0 ( ) are dependent on the value of a potential function ( 0 ) as indicated by The functional dependency of on 0 describes the stable macroscopic state that results from a pattern of continuously active synaptic interactions within and between V1 columns. This dependency also involves a control parameter. Specifically, there must be sufficient general excitation of V1 produced by action potentials that originate outside of V1 for the visual geometry to persist in the absence of electrical stimulation. As the frequency of these action potentials is increased, solutions to the equations describing the dynamics of neuron membrane potentials and action potential generation also change. At a critical point the solutions change discontinuously and a stable visual geometry pattern appears.
Although ( 0 ) is not known for the V1 network, a potential function can be introduced for a simulated neural network. The function advanced in [42] to describe the dynamics of perceptual speech categorization is modified to fit the present case as The potential landscape varies as a function of the value of as shown in Fig. 3. For = 1.5, 0 ≅ 0 and a visual geometry pattern does not exist. For = 1, ( 0 ) is relatively flat and a minimum value moving toward 0 = 1 develops. Setting = 0.5 yields a minimum within the interval 0.9996 < 0 < 1. It is proposed that is a monotonically decreasing function of the frequency of extrinsically-generated action potentials which have an excitatory effect on excitatory neurons in the neural network and that this frequency is a control parameter for emergence of the visual geometry pattern.
Sense element engagement theory asserts that visual regions and sets of vectors within and between visual regions arise through a dependence of G system strengths of interaction on a variable that behaves like visual distance. Such variables will be written without hats (e.g., ° instead of ̂°) in order to distinguish them from visual qualities. A mapping that governs G system synaptic strengths in simulations can be constructed by introducing a set of variable values on which the strengths Fig. 3. The potential landscape described by (5) and (8).
of G system synapses depend. Suppose that ⃗ = (°,°) ∈ and ⃗ = (°,°) ∈ . We form a set , = { , ,1 , … , , , , … , , , } of values that mimic visual distances between points in regions and . The values are chosen so that for each point in the presynaptic region ⃗ ∈ there exists at least one point in the postsynaptic region ⃗ ∈ such that ‖ ⃗ − ⃗ ‖ = , , for some , , ∈ , . Because of symmetry, it follows that for each point in the postsynaptic region ⃗ ∈ there exists at least one point in the presynaptic region ⃗ ∈ such that ‖ ⃗ − ⃗ ‖ = , , for some , , ∈ , . The , , will then be used to determine target values of individual synaptic strengths , , from each presynaptic neuron assigned to the column corresponding to region to each postsynaptic neuron in the column corresponding to region . For example, a piecewise function that is linear up to a maximum value ( , ) can be used to construct a simple mapping from the set of scalar variable values to the set of synaptic strengths that applies to all pairs of interacting neurons: where is the maximum synaptic strength at , , = 0. Equation (3) shows that the lightness interval distribution vanishes for very small values of 0 . Using the approach to selforganization of patterns [38] again, we state that variations in 0 ( ) are dependent on the value of a potential function ( 0 ) as indicated by and for purposes of simulations describe the dependence of on 0 as ( 0 ) = 0 − ( 0 + 1) 2 2 + ( 0 + .145) 4 4 .
The theory asserts that lightness intervals and sets of lightness differences emerge via a dependence of L system synaptic strengths on a variable that behaves like a difference in lightness values. The first step in constructing a mapping that governs L synaptic strengths in simulations involves using action potential frequencies in lightness channels in order to create variable values that behave according to (3).
In order to quantify the goodness of fit between ( , ) and 0 , a visual geometry was constructed using each of 21 equallyspaced values of 0 ranging from 0.95 to 1.05. In each geometry, the 7 regions having the smallest center-to-center Euclidean distances from each region were found and defined as nearest neighbors ( ) of that region. The maximum distance between each pair of regions was computed and ( , ) was set equal to the largest maximum value (center-tocenter distance plus the sum of the radii of each region) for each geometry. The value of was held constant at 12 . The intercept and slope parameters of the best fit linear model The close relationships between ( , ) and 0 and between | , | and 0 imply that the potential functions that characterize the size of the emergent visual geometry and the maximum contrast in the lightness interval distribution can arise by virtue of the neural network variables ( , ) ( ) and | , | ( ). If we use (6) to write as a function of visual distance ̂, , and substitute the appropriate function of 0 for ( , ) , the result describes a mapping → . Similarly, by substituting the appropriate function of 0 for | , | , we can use (9) to express as a function of absolute difference in lightness |̂, , |. Sense element engagement theory proposes that this is possible for values of the control parameter for which G and L system synaptic interactions become continuously active within and between all columns.

C. Small-Scale Neural Network Simulations that Behave in Accordance With Sense Element Engagement Theory 1) Simulation Methods
Simulations of a neural network comprised of spiking integrate-and-fire neurons interacting via conductance-based synapses illustrate key elements of the theory. A timestep of 0.05 was used in all simulations. Membrane potentials were estimated using a second-order Runge-Kutta approximation supplemented with linear interpolation of spike times. Details are provided in the appendix.  Table I. Each excitatory neuron and each inhibitory neuron in each of the 25 regions received input from each excitatory neuron in its regions (including neurons in its own region other than itself) via synapses, and from each inhibitory neuron in its regions (including the other inhibitory neuron in its own region) via synapses.
Synaptic strengths were assigned based on values of the variables , , , , ( , ) , , | , , |, and | , | . Initial values of both 0 and 0 were chosen from a uniformly distributed random variable, [0.99, 1.01]. On each timestep, the value was updated using the following Euler approximation to (4) and (7), where 0 refers to either 0 or 0 , and = 0.5. A Gaussian filter with a Gaussian kernel having radius equal to 100 was applied to each time series. The best fit linear models found in the previous section were employed to calculate values of ( , ) ( ) and of | , | ( ) from 0 ( ) and 0 ( ).
Simulated excitatory action potential input (i.e., the putative control parameter) was delivered only to excitatory network neurons. In order to avoid synchronous input to neurons in all columns, a spike train was generated independently for each of the 100 excitatory neurons, and each spike train activated 25 synapses for each neuron. The algorithm that was used for generating spike trains is described in the appendix. This same algorithm was used for adding additional spike times to both excitatory and inhibitory neurons in order to simulate the effects of electrical stimulation. Independent spike trains were generated for each excitatory neuron and for each inhibitory neuron in order to simulate the production of additional action potentials in axons having different spatial orientations and locations from a stimulating electrode [33], [34]. Results will be reported for a case in which simulated electrical stimulation producing an additional 95 spikes per second is applied to neurons in column 298.

2) Simulation Results
Results that are consistent with the sense element engagement theory will be illustrated using simulations for which = 7 and = 2 and = 0.15 (see appendix). In order to determine if extrinsically generated excitatory action potential frequency behaves as a control parameter, the proportion of columns containing neurons that produce nonzero synaptic conductance was employed as a dependent variable. This measure was used because it is assumed that continuous network-wide synaptic interactions are necessary for emergence of the visual geometry and lightness interval distribution patterns. Fig. 6(a) shows the results of simulations of 100 duration during which 1 to 10 extrinsic excitatory action potentials stimulated excitatory neurons in each column and no columns received simulated electrical stimulation. At least 4 excitatory action potentials are necessary for excitatory neurons in all columns to produce non-zero synaptic conductance, and as the number of action potentials increases the onset of network-wide active excitation occurs earlier. As shown in Fig. 6(b), inhibitory neurons produce network-wide non-zero synaptic conductance sporadically even when 10 extrinsic excitatory action potentials stimulate excitatory network neurons.
The percentage of the final 900 of 1000 -duration simulations during which network-wide active excitation and The proportion of columns containing neurons that produce non-zero inhibitory synaptic conductance is shown when 10 extrinsic, excitatory action potentials are delivered.
inhibition are present is displayed in Fig. 7. These data are displayed both in the presence (dashed lines) and absence (solid lines) of simulated electrical stimulation. The data are very similar for these two cases. As the frequency of excitatory action potentials is increased above 30/ the percentage of time during which excitatory neurons in all columns produce nonzero synaptic conductance rises rapidly, and a frequency greater than 60/ is required for network-wide excitation to be present for the final 900 of each simulation. It should also be noted that the proportion of columns producing non-zero excitatory synaptic conductance rises very quickly for larger values of the putative control parameter. When the frequency of excitatory action potentials is 90/ , the rise from a proportion of 0.0 to a proportion of 1. occurs in 4.85 in the absence of simulated electrical stimulation. Simulated electrical stimulation produces spikes that cause an immediate increase in the proportion of continuously active columns from 0.0 to 0.08, and the increase from this value to a proportion of 1. occurs in 5. 7 . The percentage of time during which all columns contain inhibitory  neurons that produce non-zero synaptic conductance rises above zero only for frequencies greater than 60/s, rises more slowly, and does not reach 100% even when the frequency of excitatory action potentials is 100/ .
The distributions of numbers of spikes that are generated by network excitatory and inhibitory neurons are displayed in Fig.  8 for a frequency of extrinsic excitatory action potentials set to 90/ . Inhibitory neurons clearly have a much greater range of frequencies and excitatory neurons have a higher mean frequency. In the absence of stimulation, the overall mean is 172.33/ for excitatory neurons and 145.16/ for inhibitory neurons. When electrical stimulation is simulated, the overall mean is 178.85/ for excitatory neurons and 144.9/ for inhibitory neurons. The very low frequencies of spikes produced by some inhibitory neurons are consistent with the lower percentage of time during which all columns contain inhibitory neurons that produce non-zero synaptic conductance.

V. A NEURAL NETWORK STRATEGY FOR TESTING SENSE ELEMENT ENGAGEMENT THEORY
Sense element engagement theory makes a number of specific claims regarding the existence of G and L systems of synapses in V1 and the dependence of strengths of interaction on quantities that behave like visual distances and absolute differences in lightness, the existence of a general excitation control parameter characterized by a critical value at which the visual geometry and lightness interval distribution patterns emerge, the existence of order parameters that characterize the visual geometry and the lightness interval distribution patterns, and the manner in which values of the order parameters depend on values of the putative control parameter. Although there are technical challenges involved in testing these claims, they are all testable in principle. However, one particular strategy for testing predictions made by the theory should be given priority because of its potential benefits.
This strategy has the goal of constructing a device that would, according to the theory, alter the visual experiences of CPV by introducing a second visual geometry. It was reported in [17] that implant recipients perceived continuous visual forms rather than patterns of discrete phosphenes when current steering was used to stimulate electrodes sequentially. According to sense element engagement theory, fusion of discrete phosphenes would occur if regions of the visual geometry pattern engaged by interacting neurons within stimulated columns overlapped. Furthermore, the theory specifies that altering regions requires changing G system strengths of interaction. It can therefore be hypothesized that continuous visual forms are perceived because current steering activates visual motion circuits which in turn modulate G system synaptic strengths in a fashion that yields the required alteration of geometric regions. This hypothesis suggests that introducing electronically-mediated interactions among populations of neurons might engage additional visual geometric regions. Two different forms of additional regions will be described.
The following considerations pertain to both forms of additional regions. It is assumed that populations in every column for which there is a corresponding electrode will be stimulated but that stimulation current that is above the threshold for producing a phosphene will be delivered through only one or a few electrodes. It is also assumed that each electrode will be used to record population activity and to stimulate on an intermittent basis. A second visual geometry should be created if the following conditions hold: (1) The amplitude of each population recording in a small time-interval determines whether an artificial action potential occurs; (2) artificial action potentials activate a set of electronic synapses for each "presynaptic and postsynaptic" pair of column populations; (3) the strengths of electronic synaptic interactions decrease as a specified function of a variable that behaves like visual distance; (4) synaptic currents modulate a signal that mimics a neuron membrane potential; and (5) the amplitude of stimulation current that is delivered to each electrode is modulated by this signal.
One possibility is to arrange the strengths of electronic synaptic interactions in order to alter the existing visual regions on which the lightness interval distribution has local maxima so that specific regions overlap. This would require that stimulation current above the threshold for producing a phosphene is delivered to each of the columns that correspond to the visual regions in which a discrete phosphene initially appears. It also requires that the extrinsic strengths of interaction are determined by a variable having values corresponding to distances among existing visual regions, among extensions to these regions that overlap, and between the overlapping extensions and the existing regions. Because lightness intervals containing high values are produced in columns that now correspond to extended, overlapping regions, a continuous visual form should result. A simulation of this possibility is shown in Fig. 1(b).
A second possibility is to add new regions of desired sizes and positions to a single, existing region on which a phosphene appears. This possibility also requires intermittent stimulation of populations of neurons in all columns but requires stimulating only one column with current that is above the threshold for creating a phosphene. The visual region that corresponds to this column will be treated as an anchor for the additional regions of desired sizes and positions. In this case, the extrinsic strengths of interaction among the populations of neurons in all columns must be determined by visual distancelike variable values among the existing visual regions, including the anchor region, among the new, desired regions, and between the existing and new regions. This possibility is illustrated by the simulation in Fig. 1(c).
Much information is required in order to construct a device that meets the conditions that are required for either form of additional regions. Having an ENN that produces lightness interval distributions on a complete visual geometry in real time would provide a platform for conducting experiments that could be very useful in selecting parameter values required for the device to perform as intended. Receding one step, it would be helpful to begin with large-scale (and computationally expensive) neural network computer simulations in order to identify parameter values (e.g., number of neurons per column, ratio of excitatory to inhibitory neurons, connectivity) that could be used to design the ENN.
Data provided by an ENN can provide a great deal of the information that is required for the recording and stimulation system. Using the ENN as a test system for research on the proposed process for producing the two forms of visual regions prior to soliciting volunteers with cortical implants would also provide a significant advantage with respect to ethical considerations. If research with the ENN provides results that are consistent with the existence of a second visual geometry and if subsequent research with human participants which is based on those results leads to subjective reports of the desired forms of visual regions, then it seems reasonable to consider seriously a claim that the ENN is a sentient device.

VI. CONCLUSION
Sense element engagement theory has been employed to develop a process for creating the visual geometry and lightness interval distribution patterns that characterize CPV and to make predictions that can be tested with human volunteers. The following steps comprise a general sense element engagement process that can be applied to other aspects of experience: 1) Identify the aspect of experience which is to be engaged by an ENN and/or modified by a device in humans. 2) Conceptualize this aspect of experience and coordinated aspects of interactions in a neural network as jointly constituting an pattern that is characterized by one or more order parameters.
3) Model the pattern using mappings → . a) Specify a system of synaptic interactions which engage elements of the pattern. b) Identify the state of the neural network that coordinates with a stable pattern. c) Identify the element of the pattern on which strengths of synaptic interactions in this system depend. d) Identify a control parameter having a critical value above which the pattern emerges. 4) Describe how the pattern emerges, i.e., its selforganization. a) Specify how temporal variations in the order parameter of the pattern depend on the value of a potential. b) Provide a mathematical description of the potential as a function of the order parameter(s) and a control parameter. c) Introduce a mapping from the values of a variable that behaves like the element of the pattern on which strengths of synaptic interactions depend. d) Identify a variable that is coordinated with the order parameter and possessing variability which accounts for the dependence of the order parameter on the value of the potential. e) Specify mappings → that emerge from the above. 5) Create a simulation of the system of synaptic interactions. 6) Use results of simulations to create a physical instantiation of the system of synaptic interactions in an ENN. 7) Predict how modification of a parameter or parameters of the neural network should produce a specified change in the aspect of human experience. 8) Use the simulations and ENN to develop a device that is predicted to modify the aspect of human experience.
The possibility of applying the sense element engagement process to other aspects of experience provides additional motivation for testing the sense element engagement theory. In particular, hardware that is based on successfully modeled aspects of human sentience could potentially provide a new approach in the development of machine intelligence [43].
APPENDIX With a few differences that will be noted, the simulations are based on equations and use parameter values provided in [44]. The sub-threshold dynamic of a neuron membrane potential is Both excitatory and inhibitory neurons were assigned a leak membrane potential = −70 , a spike threshold = −52 , and a reset potential = −59 . The leak membrane conductance was set to 25 for excitatory neurons and to 20 for inhibitory neurons. The total current ( ) is computed using conductance values at each synapse which are given by In (A2), is the peak conductance and is the reversal potential of the synapse with values listed in Table AI. The actual values of peak conductance at each synapse were obtained by multiplying by a coefficient 0 < < 1. This additional step was included so that the number of synapses from each presynaptic neuron to each postsynaptic neuron can be made large enough to provide a range of strengths that depend on values of variables that behave like visual distance and absolute difference in lightness. The synaptic function ( ) yields the time course of conductance as a difference of exponentials. When the spike threshold is exceeded producing an action potential at time * , the value of ( ) is incremented by    Table AII. Excitatory neurons were assigned an absolute refractory period of 2 and inhibitory neurons were assigned an absolute refractory period of 1 . Membrane potentials between spikes were computed using a Runge-Kutta approximation to a Taylor Series expansion that gives second-order accuracy in the discrete timestep ∆ , supplemented by linear interpolation of times * with subsequent refinement of values ( + ∆ ) as described in [45]. The Runge-Kutta approximation is given by in which computation of total conductance ( + ∆ ) is done assuming that no spike occurs between and + ∆ .
The inclusion of absolute refractory periods in simulations requires that the interpolation of values * be calculated in three different ways. The occurrence of the first spike is inferred at the first occurrence of ( ) < and ( + ∆ ) ≥ so that < * < + ∆ . Using the interpolation described in [45], the value of * is approximated as * = + ∆ − ( ) ( + ∆ ) − ( ) .
In this equation, ( + ∆ ) > denotes the value of the membrane potential at time + ∆ that has already been computed. Given that ( * ) = , the approximation and (A5) are then used to calculate a more accurate estimate of ( + ∆ ): ( + ∆ ) = + ( + ∆ − * ) ( * )⁄ Equation (A7) can also be used in the case for which at least one spike has occurred, ( + ∆ ) ≥ , and ( ) < for ≥ * + , where * stands for the time of the last action potential and denotes the absolute refractory period of the neuron.
In constructing the simulations, the control parameter for the visual geometry pattern was considered to be the number of excitatory action potentials delivered to excitatory neurons within the network in a given period of time. It was therefore decided that the algorithm employed to generate excitatory action potentials from outside of the network should create an exact number of action potentials for any specified duration of the simulation.
Suppose that the duration of the simulation is . Each excitatory network neuron has an absolute refractory period of 2 and assume that is the desired number of spikes. Each spike then occupies 2 and therefore each spike must occur within a spike interval = ( − 2 )⁄ . If the probability distribution of spike time within an interval is uniform, then the time of the first spike 1 = [0, ], 2 = [ + , 2 + ], and in general = [( − 1)( + ), + ( − 1) ] for = 1, … , where is a value drawn from a uniform distribution over the specified interval. Each time series of action potentials that was generated in this way excited excitatory network neurons in a simulated V1 column corresponding to a visual geometric region.