Previous Next Up Comments
SEHR, volume 4, issue 1: Bridging the Gap
Updated 8 April 1995

how can physical symbols act upon semiotic structures and "visualized" meanings?

Jean Petitot


Prof. Simon's paper shows very well how cognitive sciences can shed some light on difficult problems arising in textual (and in particular narrative) analysis. But perhaps it is too optimistic regarding the possibility of communication between the two cultures. For Prof. Simon-and I completely agree with his wonderful claim-"science adds to the wonders of appearance the wonders of explanation." But alas! For many literarily-oriented minds, science, as wonderful as it may be, is identified with a dreadful scientism. But nevertheless let us be optimistic (as every scientist must be) and consider Prof. Simon's main theses. I think that they are essentially right and raise many fascinating issues. I will focus here on three of them.

1. Semantics and Structural Semiotics

In what concerns semantics, the thesis is that meanings are evoked and accessed through recognition and attention processes activating a chain reaction of symbols associated with them. This associative conception of semantics is classically called "encyclopedic" and "holistic": the processes on which it rests are central (non modular) ones. It raises, therefore, all the difficult problems pointed out for instance by Jerry Fodor in his criticism of classical AI (the impossibility of working out a nomological theory of central mental processes).

Structural semiotics can be very useful here, I think, for explaining how the context can control profuse semantic "couplings" and "resonances," and select very few definite meanings in the "vast totality" of associated meanings. Even if the role of short-term memory and strongly associated contents is essential for evocation it is not sufficient. Structural analyses (Saussure, Jakobson, Hjelmslev, Lévi-Strauss, Greimas, etc.) have shown that meanings are constituted by bundles of semantic constituents-semic units traditionally called "semes." Specific components are selected by the relation of opposition between meanings. This theory of "distinctive features"-constitutive of what are called semantic paradigms-is essential for the theory of selection-by-context in

narratology. It shows that differences play the role of main cues. It is experimentally well verified and can be mathematically modeled (see Petitot, 1992). The initial and final sentences of Stendhal's La chartreuse de Parme provide a very good example. It can be shown that the "contrast" between the two political figures of Bonaparte (Caesar's and Alexander's successor) and Ernest V is the key of the book. The ideal and heroic (Napoleonian) times died at Waterloo and Fabrice's wandering tells us the story of the impossibility of becoming a hero in the new world (see Petitot, 1994b). One can also recall Lévi-Strauss' deep analysis of myths based on "paradigmatic" oppositions.

Hence the theoretical question: how can we explain the selective role of semic "contrasts" in terms of physical symbols? The problem is not trivial because, as far as we make the hypothesis that physical symbol systems implement symbolic structures which are "symbolic" in the logical sense of formal languages, we are unavoidably faced with the following difficulty: it is the characteristic lack of "paradigmatic axis" in formal languages that constitutes one of their main differences with natural ones.

2. Narrative Syntax

My second remark will concern the nature and the role of syntax in narratology (the problem of "story grammars"). In narratology, syntax does not reduce of course to classical sentence syntax. At a higher level of complexity, there exists also what is called a narrative syntax concerning the possible relations-conflict, contractual, passional relations-between characters. It is more complex than the syntax of semantic roles (Agent, Object, Beneficiary, Instrumental, etc.) which is found in case grammars. It possesses a deep anthropological dimension. For instance René Girard's relation of "mimesis" between a Subject and an Anti-Subject is such a syntactic relation. Another example is given in Stendhal's Chartreuse by the scene where Fabrice decides suddenly to leave for Waterloo after having seen in the sky an eagle flying towards Paris. It is a typical example of a syntactic relation between a Subject and what is called a "destinator" (a transcendent figure who induces the hero's destiny).

This syntax of interactions between typical narrative roles is the main device by which the semic (paradigmatic) values selected as distinctive features at the semantic level are associated with the roles and enrich progressively the characters of the story. It is a key component of narrative coherence. It yields scripts and archetypical schemas which act contextually as a sort of frame on the semantic "totality." Now the question is: how can we interpret, in the framework of the "physical symbol system hypothesis," this link between semic distinctive features (the "paradigmatic axis") and the narrative syntax (the "syntagmatic axis")?

3. "Visualized" Meanings

My third remark will be of a completely different nature and will concern the question of meanings "visualized" as mental images. There is a well known debate concerning the nature of the algorithms associated with mental images. Do they reduce to symbolic coding (classical computationalists think so) or are they of a more geometrical and dynamical nature (connectionists and many neurobiologists think so)? The problem does not concern the implementation of the algorithms but their nature (see Petitot, 1991). If the fundamental thesis-which I will call the Gestalt thesis-claiming that "a mental picture formed by retrieving some information from memory or by visualizing the meaning of a spoken or writen paragraph is stored in the same brain tissue and acted on by the same mental processes as the picture recorded by the eyes" is valid (and I think that it is effectively the case), then we are faced with a fundamental problem. Indeed, consider the algorithms relevant for computational vision: wavelet analysis, scale-space filtering and pyramidal algorithms, segmentation, Marr's 21/2D sketch, etc. They belong essentially to the mathematical universe of functional analysis, partial differential equations (e.g. anisotropic diffusion for scale-space filtering), differential geometry and dynamical systems. Their function is to construct morphological structures (from simple shapes to far more complex Gestalts).

Now several new trends in cognitive linguistics (see e.g. the works of Len Talmy, Ron Langacker, Ray Jackendoff, George Lakoff, Mark Johnson, etc.) have shown that many linguistic structures (conceptual, semantic and syntactic structures) are organized in essentially the same way as visual Gestalts. For instance, following Gruber, Ray Jackendoff claims that "in any semantic field of [EVENTS] and [STATES], the principal event-, state-, path-, and place- functions are a subset of those used for the analysis of spatial location and motion" (Jackendoff, 1983). Len Talmy studied also many linguistic "imaging systems" which constitute a "grammatically specified structuring [which] appears to be similar, in certain of its characteristics and functions, to the structuring in other cognitive domains, notably that of visual perception" (Talmy, 1978).

But if these claims are valid, and if Prof. Simon's "Gestalt thesis" is also valid, then it follows that we must henceforth model linguistic structures using mathematical models which generalize those of computational vision, and which belong therefore to the mathematical universe of dynamical systems, differential geometry, partial differential equations and functional analysis. But this means that, as far as mathematical modeling is concerned, "symbolic coding" must be separated from mathematical logic. We must, therefore, reject functionalist thesis of Fodorian type, and conceive of Gestalts (would they be visual or linguistic) as geometrical and dynamical macro-structures emerging from the underlying micro-physics. The mental symbols hypothesized by the "physical symbol system hypothesis" must be therefore conceived of in the framework of a dynamical (in the sense of dynamical systems, etc.) conception of cognitive processes (see Petitot, 1994a).

Previous Next Up Comments