http://www.bibsonomy.org/burst/user/tmalsburg/visionBibSonomy BuRST Feed for /user/tmalsburg/vision2008-07-21T01:35:08+02:00http://www.bibsonomy.org/bibtex/2abd889321fe501b8181be130419db171/tmalsburgtmalsburg2008-05-01T15:23:41+02:00changedetection dynamicfieldtheory motorcontrol workingmemory vision <span style="color:#555555;">Jeffrey S. <a href="http://www.bibsonomy.org/author/Johnson">Johnson</a> and John P. <a href="http://www.bibsonomy.org/author/Spencer">Spencer</a> and Gregor <a href="http://www.bibsonomy.org/author/Schoner">Schoner</a> </span><em>New Ideas in Psychology</em>(<em>2007</em>)Thu May 01 15:23:41 CEST 2008New Ideas in PsychologyMoving to higher ground: The dynamic field theory and the dynamics of visual cognition2007changedetection dynamicfieldtheory motorcontrol workingmemory vision In the present report, we describe a new dynamic field theory that captures the dynamics of visuo-spatial cognition. This theory grew out of the dynamic systems approach to motor control and development, and is grounded in neural principles. The initial application of dynamic field theory to issues in visuo-spatial cognition extended concepts of the motor approach to decision making in a sensori-motor context, and, more recently, to the dynamics of spatial cognition. Here we extend these concepts still further to address topics in visual cognition, including visual working memory for non-spatial object properties, the processes that underlie change detection, and the ‘binding problem’ in vision. In each case, we demonstrate that the general principles of the dynamic field approach can unify findings in the literature and generate novel predictions. We contend that the application of these concepts to visual cognition avoids the pitfalls of reductionist approaches in cognitive science, and points toward a formal integration of brains, bodies, and behavior. http://www.bibsonomy.org/bibtex/230af7cf6b221e46de3f6f4eca27900aa/tmalsburgtmalsburg2007-12-10T12:10:27+01:00vision scanpaths editdistance attention perception <span style="color:#555555;">Sheree <a href="http://www.bibsonomy.org/author/Josephson">Josephson</a> and Michael E. <a href="http://www.bibsonomy.org/author/Holmes">Holmes</a> </span><em>Proceedings of the symposium on Eye tracking research \&amp; applications</em>(<em>2002</em>)Mon Dec 10 12:10:27 CET 2007Proceedings of the symposium on Eye tracking research \& applications43--49{Visual attention to repeated internet images: testing the scanpath theory on the world wide web}2002vision scanpaths editdistance attention perception The somewhat controversial and often-discussed theory of visual perception, that of scanpaths, was tested using Web pages as visual stimuli. In 1971, Noton and Stark defined "scanpaths" as repetitive sequences of fixations and saccades that occur upon re-exposure to a visual stimulus, facilitating recognition of that stimulus. Since Internet users are repeatedly exposed to certain visual displays of information, the Web is an ideal stimulus to test this theory. Eye-movement measures were recorded while subjects repeatedly viewed three different kinds of Internet pages -- a portal page, an advertising page and a news story page -- over the course of a week. Scanpaths were compared by using the string-edit methodology that measures resemblance between sequences. Findings show that on the World Wide Web, with somewhat complex visual digital images, some viewers' eye movements may follow a habitually preferred path -- a scanpath -- across the visual display. In addition, strong similarity among eye-path sequences of different viewers may indicate that other forces such as features of the Web site or memory are important.Excitatory synaptic inputs to spiny stellate cells...[Nature. 1996] - PubMed Resulthttp://www.bibsonomy.org/bibtex/2c10062af9d4bea60635c0e0ebdb0099b/tmalsburgtmalsburg2007-11-04T18:55:24+01:00cellular -23PDrivingModulating cortex vision <span style="color:#555555;">K J <a href="http://www.bibsonomy.org/author/Stratford">Stratford</a> and K <a href="http://www.bibsonomy.org/author/Tarczy-Hornoch">Tarczy-Hornoch</a> and K A <a href="http://www.bibsonomy.org/author/Martin">Martin</a> and N J <a href="http://www.bibsonomy.org/author/Bannister">Bannister</a> and J J <a href="http://www.bibsonomy.org/author/Jack">Jack</a> </span><em>Nature</em><em>382(6588):258-261</em><em>Jul1996. </em>Sun Nov 04 18:55:24 CET 2007NatureJul6588258-261Excitatory synaptic inputs to spiny stellate cells in cat visual cortex3821996cellular -23PDrivingModulating cortex vision In layer 4 of cat visual cortex, the monocular, concentric receptive fields of thalamic neurons, which relay retinal input to the cortex, are transformed into 'simple' cortical receptive fields that are binocular and selective for the precise orientation, direction of motion, and size of the visual stimulus. These properties are thought to arise from the pattern of connections from thalamic neurons, although anatomical studies show that most excitatory inputs to layer 4 simple cells are from recurrently connected circuits of cortical neurons. We examined single fibre inputs to spiny stellate neurons. We examined single fibre inputs to spiny stellate neurons in slices of cat visual cortex, and conclude that thalamocortical synapses are powerful and the responses they evoke are unusually invariant for central synapses. However, the responses to intracortical inputs, although less invariant, are strong enough to provide most of the excitation to simple cells in vivo. Our results suggest that the recurrent excitatory circuits of cortex may amplify the initial feedforward thalamic signal, subserving dynamic modifications of the functional properties of cortical neurons.Magnocellular and parvocellular contributions to t...[J Neurosci. 1994] - PubMed Resulthttp://www.bibsonomy.org/bibtex/280420824f3139a8d8aef462d98e95ffc/tmalsburgtmalsburg2007-11-04T18:50:57+01:00vision cortex -23PDrivingModulating cellular <span style="color:#555555;">T A <a href="http://www.bibsonomy.org/author/Nealey">Nealey</a> and J H <a href="http://www.bibsonomy.org/author/Maunsell">Maunsell</a> </span><em>J Neurosci</em><em>14(4):2069-2079</em><em>Apr1994. </em>Sun Nov 04 18:50:57 CET 2007J NeurosciApr42069-2079Magnocellular and parvocellular contributions to the responses of neurons in macaque striate cortex141994vision cortex -23PDrivingModulating cellular Anatomical and physiological studies of the primate visual system have suggested that the signals relayed by the magnocellular and parvocellular subdivisions of the LGN remain segregated in visual cortex. It has been suggested that this segregation may account for the known differences in visual function between the parietal and temporal cortical processing streams in extrastriate visual cortex. To test directly the hypothesis that the temporal stream of processing receives predominantly parvocellular signals, we recorded visual responses from the superficial layers of V1 (striate cortex), which give rise to the temporal stream, while selectively inactivating either the magnocellular or parvocellular subdivisions of the LGN. Inactivation of the parvocellular subdivision reduced neuronal responses in the superficial layers of V1, but the effects of magnocellular blocks were generally as pronounced or slightly stronger. Individual neurons were found to receive contributions from both pathways. We furthermore found no evidence that magnocellular contributions were restricted to either the cytochrome oxidase blobs or interblobs in V1. Instead, magnocellular signals made substantial contributions to responses throughout the superficial layers. Thus, the regions within V1 that constitute the early stages of the temporal processing stream do not appear to contain isolated parvocellular signals. These results argue against a direct mapping of the subcortical magnocellular and parvocellular pathways onto the parietal and temporal streams of processing in cortex.Microsaccades uncover the orientation of covert at...[Vision Res. 2003] - PubMed Resulthttp://www.bibsonomy.org/bibtex/2a6ac7fcc4994e695e649edf9b2ba4505/tmalsburgtmalsburg2007-10-30T17:38:06+01:00vision eyemovements algorithm saccades B_scanpathsimilarity <span style="color:#555555;">Ralf <a href="http://www.bibsonomy.org/author/Engbert">Engbert</a> and Reinhold <a href="http://www.bibsonomy.org/author/Kliegl">Kliegl</a> </span><em>Vision Res</em><em>43(9):1035-1045</em><em>Apr2003. </em>Tue Oct 30 17:38:06 CET 2007Vision ResApr91035-1045Microsaccades uncover the orientation of covert attention432003vision eyemovements algorithm saccades B_scanpathsimilarity Fixational eye movements are subdivided into tremor, drift, and microsaccades. All three types of miniature eye movements generate small random displacements of the retinal image when viewing a stationary scene. Here we investigate the modulation of microsaccades by shifts of covert attention in a classical spatial cueing paradigm. First, we replicate the suppression of microsaccades with a minimum rate about 150 ms after cue onset. Second, as a new finding we observe microsaccadic enhancement with a maximum rate about 350 ms after presentation of the cue. Third, we find a modulation of the orientation towards the cue direction. These multiple influences of visual attention on microsaccades accentuate their role for visual information processing. Furthermore, our results suggest that microsaccades can be used to map the orientation of visual attention in psychophysical experiments.Feedback connections and conscious vision. [Trends Cogn Sci. 2001] - PubMed Resulthttp://www.bibsonomy.org/bibtex/2d9283bee23d5113b1db7466a5c83b5f2/tmalsburgtmalsburg2007-10-30T09:43:24+01:00vision feedbackconnections tms consciousness <span style="color:#555555;">J <a href="http://www.bibsonomy.org/author/Bullier">Bullier</a> </span><em>Trends Cogn Sci</em><em>5(9):369-370</em><em>Sep2001. </em>Tue Oct 30 09:43:24 CET 2007Trends Cogn SciSep9369-370Feedback connections and conscious vision52001vision feedbackconnections tms consciousness A common network of functional areas for attention...[Neuron. 1998] - PubMed Resulthttp://www.bibsonomy.org/bibtex/2131304ad8223903525eac5d2f5a4c880/tmalsburgtmalsburg2007-10-29T18:54:10+01:00fmri vision -23PAttention feedback eyemovements attention braintopology <span style="color:#555555;">M <a href="http://www.bibsonomy.org/author/Corbetta">Corbetta</a> and E <a href="http://www.bibsonomy.org/author/Akbudak">Akbudak</a> and T E <a href="http://www.bibsonomy.org/author/Conturo">Conturo</a> and A Z <a href="http://www.bibsonomy.org/author/Snyder">Snyder</a> and J M <a href="http://www.bibsonomy.org/author/Ollinger">Ollinger</a> and H A <a href="http://www.bibsonomy.org/author/Drury">Drury</a> and M R <a href="http://www.bibsonomy.org/author/Linenweber">Linenweber</a> and S E <a href="http://www.bibsonomy.org/author/Petersen">Petersen</a> and M E <a href="http://www.bibsonomy.org/author/Raichle">Raichle</a> and D C Van <a href="http://www.bibsonomy.org/author/Essen">Essen</a> and G L <a href="http://www.bibsonomy.org/author/Shulman">Shulman</a> </span><em>Neuron</em><em>21(4):761-773</em><em>Oct1998. </em>Mon Oct 29 18:54:10 CET 2007NeuronOct4761-773A common network of functional areas for attention and eye movements211998fmri vision -23PAttention feedback eyemovements attention braintopology Functional magnetic resonance imaging (fMRI) and surface-based representations of brain activity were used to compare the functional anatomy of two tasks, one involving covert shifts of attention to peripheral visual stimuli, the other involving both attentional and saccadic shifts to the same stimuli. Overlapping regional networks in parietal, frontal, and temporal lobes were active in both tasks. This anatomical overlap is consistent with the hypothesis that attentional and oculomotor processes are tightly integrated at the neural level.http://www.bibsonomy.org/bibtex/27a40bd0d9c3abd515602e4611d20891e/tmalsburgtmalsburg2007-10-29T17:10:32+01:00fmri ekp vision attention -23PAttention feedbackconnections primaryvisualcortex <span style="color:#555555;">A. <a href="http://www.bibsonomy.org/author/Martínez">Mart&#237;nez</a> and L. <a href="http://www.bibsonomy.org/author/Anllo-Vento">Anllo-Vento</a> and M. I. <a href="http://www.bibsonomy.org/author/Sereno">Sereno</a> and L. R. <a href="http://www.bibsonomy.org/author/Frank">Frank</a> and R. B. <a href="http://www.bibsonomy.org/author/Buxton">Buxton</a> and D. J. <a href="http://www.bibsonomy.org/author/Dubowitz">Dubowitz</a> and E. C. <a href="http://www.bibsonomy.org/author/Wong">Wong</a> and H. <a href="http://www.bibsonomy.org/author/Hinrichs">Hinrichs</a> and H. J. <a href="http://www.bibsonomy.org/author/Heinze">Heinze</a> and S. A. <a href="http://www.bibsonomy.org/author/Hillyard">Hillyard</a> </span><em>Nature Neuroscience</em>(<em>1999</em>)Mon Oct 29 17:10:32 CET 2007Nature Neuroscience364 - 369Involvement of striate and extrastriate visual cortical areas in spatial attention1999fmri ekp vision attention -23PAttention feedbackconnections primaryvisualcortex We investigated the cortical mechanisms of visual-spatial attention while subjects discriminated patterned targets within distractor arrays. Functional magnetic resonance imaging (fMRI) was used to map the boundaries of retinotopic visual areas and to localize attention-related changes in neural activity within several of those areas, including primary visual (striate) cortex. Event-related potentials (ERPs) and modeling of their neural sources, however, indicated that the initial sensory input to striate cortex at 50−55 milliseconds after the stimulus was not modulated by attention. The earliest facilitation of attended signals was observed in extrastriate visual areas, at 70−75 milliseconds. We hypothesize that the striate cortex modulation found with fMRI may represent a delayed, re-entrant feedback from higher visual areas or a sustained biasing of striate cortical neurons during attention. ERP recordings provide critical temporal information for analyzing the functional neuroanatomy of visual attention.Cortical magnification factor predicts the photopi...[Nature. 1978] - PubMed Resulthttp://www.bibsonomy.org/bibtex/275be7ac875374ccc0cde886e658e7529/tmalsburgtmalsburg2007-09-29T18:00:55+02:00primaryvisualcortex eye vision B_scanpathsimilarity perception psychophysics <span style="color:#555555;">J <a href="http://www.bibsonomy.org/author/Rovamo">Rovamo</a> and V <a href="http://www.bibsonomy.org/author/Virsu">Virsu</a> and R <a href="http://www.bibsonomy.org/author/Näsänen">N&#228;s&#228;nen</a> </span><em>Nature</em><em>271(5640):54-56</em><em>Jan1978. </em>Sat Sep 29 18:00:55 CEST 2007NatureJan564054-56Cortical magnification factor predicts the photopic contrast sensitivity of peripheral vision2711978primaryvisualcortex eye vision B_scanpathsimilarity perception psychophysics Human photoreceptor topography. [J Comp Neurol. 1990] - PubMed Resulthttp://www.bibsonomy.org/bibtex/29ca2423e68ca46f128261f63352c0cdf/tmalsburgtmalsburg2007-09-27T14:49:57+02:00B_scanpathsimilarity physiology vision retina neurobiology perception <span style="color:#555555;">C. A. <a href="http://www.bibsonomy.org/author/Curcio">Curcio</a> and K. R. <a href="http://www.bibsonomy.org/author/Sloan">Sloan</a> and R. E. <a href="http://www.bibsonomy.org/author/Kalina">Kalina</a> and A. E. <a href="http://www.bibsonomy.org/author/Hendrickson">Hendrickson</a> </span><em>Journal of Comparative Neurology</em><em>292(4):497-523</em><em>Feb1990. </em>Thu Sep 27 14:49:57 CEST 2007Journal of Comparative NeurologyFeb4497-523Human photoreceptor topography2921990B_scanpathsimilarity physiology vision retina neurobiology perception We have measured the spatial density of cones and rods in eight whole-mounted human retinas, obtained from seven individuals between 27 and 44 years of age, and constructed maps of photoreceptor density and between-individual variability. The average human retina contains 4.6 million cones (4.08-5.29 million). Peak foveal cone density averages 199,000 cones/mm2 and is highly variable between individuals (100,000-324,000 cones/mm2). The point of highest density may be found in an area as large as 0.032 deg2. Cone density falls steeply with increasing eccentricity and is an order of magnitude lower 1 mm away from the foveal center. Superimposed on this gradient is a streak of high cone density along the horizontal meridian. At equivalent eccentricities, cone density is 40-45% higher in nasal compared to temporal retina and slightly higher in midperipheral inferior compared to superior retina. Cone density also increases slightly in far nasal retina. The average human retina contains 92 million rods (77.9-107.3 million). In the fovea, the average horizontal diameter of the rod-free zone is 0.350 mm (1.25 degrees). Foveal rod density increases most rapidly superiorly and least rapidly nasally. The highest rod densities are located along an elliptical ring at the eccentricity of the optic disk and extending into nasal retina with the point of highest density typically in superior retina (5/6 eyes). Rod densities decrease by 15-25% where the ring crosses the horizontal meridian. Rod density declines slowly from the rod ring to the far periphery and is highest in nasal and superior retina. Individual variability in photoreceptor density differs with retinal region and is similar for both cones and rods. Variability is highest near the fovea, reaches a minimum in the midperiphery, and then increases with eccentricity to the ora serrata. The total number of foveal cones is similar for eyes with widely varying peak cone density, consistent with the idea that the variability reflects differences in the lateral migration of photoreceptors during development. Two fellow eyes had cone and rod numbers within 8% and similar but not identical photoreceptor topography.Letter: A chart demonstrating variations in acuity...[Vision Res. 1974] - PubMed Resulthttp://www.bibsonomy.org/bibtex/20d2868f70a1b72a06a36ba6ecae21a12/tmalsburgtmalsburg2007-09-27T14:48:17+02:00psychophysics retina perception vision corticalmagnification B_scanpathsimilarity <span style="color:#555555;">S. M. <a href="http://www.bibsonomy.org/author/Anstis">Anstis</a> </span><em>Vision Research</em><em>14(7):589-592</em><em>Jul1974. </em>Thu Sep 27 14:48:17 CEST 2007Vision ResearchJul7589-592Letter: A chart demonstrating variations in acuity with retinal position141974psychophysics retina perception vision corticalmagnification B_scanpathsimilarity Visual resolution, contrast sensitivity, and the c...[Exp Brain Res. 1979] - PubMed Resulthttp://www.bibsonomy.org/bibtex/2bfb829b1f3c225f159dfac4b73df6368/tmalsburgtmalsburg2007-09-27T14:44:55+02:00corticalmagnification primaryvisualcortex vision neurobiology B_scanpathsimilarity perception <span style="color:#555555;">V. <a href="http://www.bibsonomy.org/author/Virsu">Virsu</a> and J. <a href="http://www.bibsonomy.org/author/Rovamo">Rovamo</a> </span><em>Experimental Brain Research</em><em>37(3):475-494</em>(<em>1979</em>)Thu Sep 27 14:44:55 CEST 2007Experimental Brain Research3475-494Visual resolution, contrast sensitivity, and the cortical magnification factor371979corticalmagnification primaryvisualcortex vision neurobiology B_scanpathsimilarity perception This study shows that photopic contrast sensitivity and resolution can be predicted by means of simple functions derived by using the cortical magnification factor M as a scale factor of mapping from the visual field into the striate cortex. We measured the minimum contrast required for discriminating the direction of movement or orientation of sinusoidal gratings, or for detecting them in central and peripheral vision. No qualitative differences were found between central and peripheral vision, and almost all quantitative differences observed could be removed by means of a size compensation derived from M. The results indicated specifically that (1) visual patterns can be made equally visible if they are scaled so that their calculated cortical representations become equivalent; (2) contrast sensitivity follows the same power function of the cortical area stimulated by a grating at any eccentricity; (3) area and squared spatial frequency are reciprocally related as determinants of contrast sensitivity; and (4) acuity and resolution are directly proportional to M, and the minimum angle of resolution is directly proportional to M-1. The power law of spatial summation expressed in (2) and (3) suggests the existence of a central integrator that pools the activity of cortical neurons. This summation mechanism makes the number of potentially activated visual cells the most important determinant of visibility and contrast sensitivity. The functional homogeneity of image processing across the visual field observed here agrees with the assumed anatomical and physiological uniformity of the visual cortex.Visual resolution and contour interaction in the f...[Vision Res. 1979] - PubMed Resulthttp://www.bibsonomy.org/bibtex/21b75e43cdb9a1100d150e51c456ac96e/tmalsburgtmalsburg2007-09-27T14:42:07+02:00B_scanpathsimilarity vision preception physiology neurobiology retina <span style="color:#555555;">R. J. <a href="http://www.bibsonomy.org/author/Jacobs">Jacobs</a> </span><em>Vision Research</em><em>19(11):1187-1195</em>(<em>1979</em>)Thu Sep 27 14:42:07 CEST 2007Vision Research111187-1195Visual resolution and contour interaction in the fovea and periphery191979B_scanpathsimilarity vision preception physiology neurobiology retina The representation of the visual field on the cere...[J Physiol. 1961] - PubMed Resulthttp://www.bibsonomy.org/bibtex/2cb94019c1e7ee744c387a8884cf692a5/tmalsburgtmalsburg2007-09-27T14:34:17+02:00visualsystem B_scanpathsimilarity neurobiology vision perception physiology <span style="color:#555555;">P. M. <a href="http://www.bibsonomy.org/author/Daniel">Daniel</a> and D. <a href="http://www.bibsonomy.org/author/Whitteridge">Whitteridge</a> </span><em>The Journal of Physiology</em><em>Dec1961. </em>Thu Sep 27 14:34:17 CEST 2007The Journal of PhysiologyDec203-221The representation of the visual field on the cerebral cortex in monkeys1591961visualsystem B_scanpathsimilarity neurobiology vision perception physiology On the basis of his extensive and elegant anatomical investigations on the visual cortex, Poliak (1932) suggested that a mathematical projection of the retina on the cerebral cortex must exist. Talbot & Marshall (1941) used physiological methods to map the central part of the visual field on to the posterolateral surface ofthe cortex in the monkey. They devised an index of cortical representation expressed as the increment of the angle, measured radially from the centre of gaze, which is represented on each millimetre of cortex. We have confirmed their observations and have extended the mapp- ing to the buried visual cortex in the horizontal and vertical calcarine fissures. We have preferred to use the reciprocal of their index and to call it the cortical magnification, M. When this is measured along radii and at right angles to them, it provides the empirical quantitative relation which Polyak wanted. It also defines the shape and size of the visual receptive field. We have made such a surface, folded it and compared it with the calcarine cortex of the monkey. A preliminary account of this work has been published (Daniel & Whitteridge, 1959). http://www.bibsonomy.org/bibtex/2c1de452a8b17cc0247c29eb1f771b09d/tmalsburgtmalsburg2007-01-22T14:39:55+01:00language grounding machinelearning vision multimodality <span style="color:#555555;">Brian E. <a href="http://www.bibsonomy.org/author/Pangburn">Pangburn</a> and Robert C. <a href="http://www.bibsonomy.org/author/Mathews">Mathews</a> and S. <a href="http://www.bibsonomy.org/author/Sitharama Iyengar">Sitharama Iyengar</a> and Jonathan P. <a href="http://www.bibsonomy.org/author/Ayo">Ayo</a> </span><em>Proceedings of the HLT-NAACL 2003 workshop on Learning word meaning from non-linguistic data, </em><em>page46--53. </em><em>Morristown, NJ, USA, </em><em>Association for Computational Linguistics, </em>(<em>2003</em>)Mon Jan 22 14:39:55 CET 2007Morristown, NJ, USAProceedings of the HLT-NAACL 2003 workshop on Learning word meaning from non-linguistic data46--53EBLA: A Perceptually Grounded Model of Language Acquisition2003language grounding machinelearning vision multimodality This paper introduces an open computational framework for visual perception and grounded language acquisition called Experience-Based Language Acquisition (EBLA). EBLA can "watch" a series of short videos and acquire a simple language of nouns and verbs corresponding to the objects and object-object relations in those videos. Upon acquiring this protolanguage, EBLA can perform basic scene analysis to generate descriptions of novel videos. The performance of EBLA has been evaluated based on accuracy and speed of protolanguage acquisition as well as on accuracy of generated scene descriptions. For a test set of simple animations, EBLA had average acquisition success rates as high as 100% and average description success rates as high as 96.7%. For a larger set of real videos, EBLA had average acquisition success rates as high as 95.8% and average description success rates as high as 65.3%. The lower description success rate for the videos is attributed to the wide variance in the appearance of objects across the test set. While there have been several systems capable of learning object or event labels for videos, EBLA is the first known system to acquire both nouns and verbs using a grounded computer vision system. Learning Visually-Grounded Words and Syntax for a Scene Description Task - Roy (ResearchIndex)http://www.bibsonomy.org/bibtex/22692e9d34027345b0308f7bd974b37b5/tmalsburgtmalsburg2007-01-22T14:00:43+01:00multimodality grounding machinelearning vision semantics <span style="color:#555555;">Deb K. <a href="http://www.bibsonomy.org/author/Roy">Roy</a> </span>(<em>2002</em>) <em>Computer Speech and Language, In review. . </em>Mon Jan 22 14:00:43 CET 2007Computer Speech and Language, In review.Learning Visually-Grounded Words and Syntax for a Scene Description Task2002multimodality grounding machinelearning vision semantics A spoken language generation system has been developed that learns to describe objects in computer-generated visual scenes. The system is trained by a `show-and-tell' procedure in which visual scenes are paired with natural language descriptions. Learning algorithms acquire probabilistic structures which encode the visual semantics of phrase structure, word classes, and individual words. Using these structures, a planning algorithm integrates syntactic, semantic, and contextual constraints to generate natural and unambiguous descriptions of objects in novel scenes. The system generates syntactically well-formed compound adjective noun phrases, as well as relative spatial clauses. The acquired linguistic structures generalize from training data, enabling the production of novel word sequences which were never observed during training. The output of the generation system is synthesized using word-based concatenative synthesis drawing from the original training speech corpus. In evaluations of semantic comprehension by human judges, the performance of automatically generated spoken descriptions was comparable to human generated descriptions. This work is motivated by our long term goal of developing spoken language processing systems which grounds semantics in machine perception and action. Representation of Human Vision in the Brain: How Does Human Perception Recognize Images?http://www.bibsonomy.org/bibtex/23c9789092ad2f71d2f8b251c0f8272a6/tmalsburgtmalsburg2006-12-20T23:11:42+01:00clustering scanpath vision eyemovements cognitivescience <span style="color:#555555;">Lawrence W. <a href="http://www.bibsonomy.org/author/Stark">Stark</a> and Claudio M. <a href="http://www.bibsonomy.org/author/Privitera">Privitera</a> and Huiyang <a href="http://www.bibsonomy.org/author/Yang">Yang</a> and Michela <a href="http://www.bibsonomy.org/author/Azzariti">Azzariti</a> and Yeuk Fai <a href="http://www.bibsonomy.org/author/Ho">Ho</a> and Ted <a href="http://www.bibsonomy.org/author/Blackmon">Blackmon</a> and Dimitri <a href="http://www.bibsonomy.org/author/Chernyak">Chernyak</a> </span><em>Journal of Electronic Imaging</em><em>January2001. </em>Wed Dec 20 23:11:42 CET 2006Journal of Electronic Imaging January123­--151 Representation of Human Vision in the Brain: How Does Human Perception Recognize Images?2001clustering scanpath vision eyemovements cognitivescience The repetitive scanpath eye movement, EM, sequence enabled an approach to the representation of visual images in the human brain. We supposed that there were several levels of binding--semantic or symbolic binding; structural binding for the spatial locations of the regions-of-interest; and sequential binding for the dynamic execution program that yields the sequence of EMs. The scanpath sequences enable experimental evaluation of these various bindings that appear to play independent roles and are likely located in different parts of the modular cortex. EMs play an essential role in top-down control of the flow of visual information. The scanpath theory proposes that an internal spatial-cognitive model controls perception and the active looking EMs. Evidence supporting the scanpath theory includes experiments with ambiguous figures, visual imagery, and dynamic scenes. It is further explicated in a top-down computer vision tracking scheme for telerobots using design elements from the scanpath procedures. We also introduce procedures--calibration of EMs, identification of regions-of-interest, and analysis and comparison programs--for studying scanpaths. Although philosophers have long speculated that we see in our mind's eye, yet until the scanpath theory, no strong scientific evidence was available to support these conjectures.