Art and Computer

Abraham Moles (translated from the original by Eamonn Bell c. 2023)

The following pages are extracted from a book by Abraham Moles, Art et ordinateur, which will be published before the end of the year [1970]. We have already spoken in previous issues about computational artistic creation. But this chapter adopts a more general perspective. Without entering the field of any one art in particular, it envisages the general conditions of artistic creation by means of cybernetics, whatever the field of application. It thus renews the very notion of aesthetics, envisaging the new role that it could be called upon to play.

Artificial and cybernetic creation. From analogy to simulation

Advances in computing have led to the creation of some rough drafts of artificial creation. A number of processes already belong to the field of the machine; it is interesting to verify the part they play in the main phenomenon: systematic variations on model, addition of supplementary elements of limited variety (decoration), systematic exploration of a field of possibilities (permutational art), combinatorics, etc.

A machine is perfectly capable of emitting series of numbers or words, one after the other; that is to say it can indulge in the smearing of the chimpanzee painter or the smearing of the automatic discourse. It can also reproduce indefinitely the discourse of the parrot. But it is certain that it is unable to reconstruct a tragedy of Racine (or even of R.A.C.N.A.C.) (Rhythm autogenerator, correlator, integrator, notionalyser and computer).

These complexity-handling machines process information according to certain simulation models that are a kind of materialized “Gedankenexperiment”. Computer programming has many similarities to the pattern of cybernetic reasoning: the latter is based on the discovery of an analogy and imposes a number of restrictive conditions on it before considering it as a model for an effective simulation. The cybernetic method of analogies suggests us to reproduce as perfectly as possible, by a sort of iteration of thought, all the processes that we master, to simulate all that is possible to simulate.

At the same time, it criticizes the shortcomings of the model, in the order in which they appear, so as to improve it by “trial and error”. It makes it possible to clearly define, at the end of the analysis, the doctrinal residue that we come up against; we then resort to other methods. It is therefore the most rational approach possible, even if logically this undertaking seems desperate: “It is not necessary to hope to start, but to undertake to persevere.” ??

In the field of machine translation, we believed a few years ago that the simulation of human thought by machines would progress infinitely faster, and that their deficient abilities would be rapidly improved. The psychological profile that can be traced leads to an error of assessment. Thought is materialized by signs, meaningless for the computer, which cannot participate in the mental image these signs evoke. Translation is not a simple transfer from one sign system to another, but rather it aims to restore a sequence of ideas, to communicate as faithfully as possible to a receiver of a language Y the mental representation of a transmitter of a language X. The words form an arbitrary system and reflect according to the languages a different conceptual division of reality. A text is only mathematically transcribable if the concepts are clearly defined and the contextual information is made explicit. Otherwise, the computer cannot transmit any meaning.

The apprehension of concepts by machines - creation in the broad sense - remains in a state of becoming and we have strong reasons to believe that we are facing very fundamental difficulties on which we risk spending a lot of time (Godel’s Theorem). The creation of valid, meaningful artificial texts is the “core problem” (Kernproblem): it is that of the ability of machines to master a “general semiotic” or science of signs.

It is here that the construction of the work of art by machine asserts its interest. Indeed, the machine claims to approach the work of art in all possible ways, and, each time, to propose simulacra of it, each characteristic of a different conception of the work. The degree of similarity plays the role of the old value “Truth”. It is not wrong to call this method “neo-Cartesian,” a machinic neo-Cartesianism based on operationality.

Yet it turns out that artificial creation has, in aesthetics, conditions for the validity of its products which are quite different from those which the finished scientific product should have, conditions which seem quite a lot easier to fulfil. It then appears that, in the many versions which the “creative machine” can offer, it is the versions of an aesthetic nature which would be the easiest to produce quickly. Consequently, according to the constant rule of the scientific approach which begins with the easiest, these would represent the best use of necessarily limited means. In other words: the mechanisms of artistic creation appear as a rough draft, which can be achieved, wherease the mechanisms of scientific creation—where the same basic problems are posed—have requirements of acceptance that are more difficult to achieve materially.

On the degrees between the single atom and the total: the proximate order and the distal order

The extensive work done in linglinguisticsuistics on the structure of language and the possibility of giving and account of it through “models” allows us to clarify this assertion a little more. Indeed, the structuralist hypothesis, which consists in breaking down the world into atoms of perception in order to reorganize them according to certain rules, the whole of which constitutes the “structures”, originates precisely from linguistics.

But it has recently invaded the whole of the human sciences through the Theory of Information, which synthesizes the atomistic attitude of structure and the dialectical attitude of totality. The dialectical rhetoric has revealed the basis for a classification of the laws of constraints governing the assembly of the elements to constitute a “form”, that is to say a total perceivable by the mind; the forms do not exist in themselves, they are only “perceived”: they are products of the receiver, applied to the message.

It is therefore necessary to classify the rules of constraint from a new perspective: that of the receiver. This dimension delineates the fundamental opposition between proximate order and distal order.

Mathematically, this opposition can be characterized by the concept of autocorrelation distance. Reduced to its essence, autocorrelation means a statistical measure of the average influence on any element of a message or object by the existence of another element of the message, located at a given distance.

In the proximate order, the closer the observer gets to the elements of the system he observes, the better the links of association between these elements appear to him. It is the local aspects that interest him; it is only these that are clear and obvious: microscopic study will account for this. But the more or less considerable disorder of the elements can conceal the general structure.

In the distal order, on the contrary, the further the observer is from the phenomenon he observes, the better he understands the general structure, the overall organization. The overall forms emerge as a master plan, governing the atoms of perception and integrating them into a hierarchy. The observer forgets, or neglects, the local fluctuations, even when they are so considerable that they drown out, at short range, the global function.

Proximate and distal order are each measured by a degree of order which may be more or less elevated, and on which mathematicians exercise their wisdom. It can of course happen that the order is both proximate and distal: we will then say that there is a total order. In fact, in the dialectical play between the banal and the original that characterizes the artistic or scientific work of the human mind, the quantities of order between the extremes are quite different.

Let us take a text, for example, with the ambition of creating a “machine for creating texts”.

A patent for an invention - which is a text … - essentially obeys a long-range order structure, namely its coherence of reasoning. It very strictly regulates the order of the paragraphs, the choice of words, formulas, expressions and their juxtaposition (by a quasi-deductive law, which is a matter of logic); this structure is re-masted in a very banal syntax by the writer who lets himself go with his habits, uses mouth-hole words and writes texts of an approximate correlation.

On the contrary, in a text of modern poetry, as Paul Valéry’s fine analyses had shown very clearly; poets like Isou and Queneau have similarly experimented. Dufrêne, Lambert, after the surrealist attempts of “automatic writing”, what matters most is the structure of close order, the association of words, the exploitation of the rules of association by neighborhood. William James had already shed substantial light on these.

The distal order also exists in poetry. Take a “figurative” poet, Hugo, for example, telling a moving story of a soldier wounded on the battlefield: the whole poem converges towards the last sentence. Our interest in this distal structure is relatively independent of the former: the impeccable progress of a French syntax is, so to speak, optional; the experimenters did not fail to put it to the test, with sometimes remarkable results.

Here, the contrast between proximate and distal order, treated very differently in scientific and artistic work clearly reappears. However, it turns out that, in the development of work on “machines for organising the sequences of elements drawn from a repertoire” — which could be a developed and rigorous name for the somewhat journalistic term “creative machine” — our progress in the domain of the proximate order is much greater than that achieved in that of the distal order.

The proximate order is articulated, in information theory, as di-, tri- or polygrammatic processes, or even Markov processes: it is governed by transition matrices with 2, 3, or n dimensions. But the construction of texts by more and more extensive Markovian approximations has revealed the limits of this operation. By definition, the larger the size of the matrices, the larger they are (order n), and the less valid they are. What’s more, our current machines show a deliberate reluctance to manipulate data that is too large, the cost having some relation to do with the amount of storage.

The distal order, for its part, is characterized by the intervention of certain types of laws, such as grammatical correction, subordination of one sentence to another, logical continuity or continuity of gaze, everything that linguists currently group under the name of “syntactic structures”. Now the syntactic structures of the work of art are currently poorly known from the scientific point of view: a critic who speaks, in an Italian painting, of the domination of the illumination of the Virgin over all the characters that are found around her, speaks indeed of forms belonging to the syntactic structures of the work, but he does little but describe them and does not set forth for them any scientific rule, that is to say a rule that is mathematical. In fact, we know very little in this area, and we are extremely inapt at fitting the little we know into analogical models.

The best approximations were, on the one hand, those made by Chomsky in his studies on language and, on the other hand, all the work grouped around the concepts of “supersign” and “supersign hierarchy”. In short, we are unable to manipulate long-range order with sufficient rigour. This is, among other reasons, one of the reasons for our failure in the field of machines for creating scientific texts. The situation of artistic creation, on the other hand, appears to be much more favourable. It is already possible to produce musical, pictorial or poetic works in which the respective shares of proximate and distal order are sufficient, at least to satisfy the popular consensus of the definition of the work of art; in other words, that it be consumable. It then appears to be a good policy for a society, whose means are not unlimited, to deal with its fundamental problems — fertilization and renewal are top in the order of the speediest profitability, that is to say to start with artistic creation. The experience gained in this field will be easily transferable to the field of scientific creation.

Cybernetic analysis of aesthetic attitudes. A science of creation

This shaping of the creative function in the “affluent” society gives a new role to the aesthetician, who is no longer the ethereal philosopher speaking about the Beautiful, but the practitioner of sensations, solidly trained in the psychology of values and preparing the work of the “translation machine”. Experiments in this area have revealed some fundamental attitudes that are amenable to classification. Each is an aesthetic attitude, which we are free to symbolize by a type of flowchart, each type representing a program of a machine to be created. Cybernetic analysis, by quantifying the attitudes it demands, already brings something original about the nature of creation.

First attitude

Aesthetics, the critic of Nature: the machine spectator or artificial explores the beauties of the natural world and carries out a statistical characterization.

The world is full of wealth and it must be exploited. This amounts to transforming, according to criteria of originality, any system of exploration of the world into a system of valorization. The machine would include at its input a translator of sensations into machine language (examples: television camera, artificial ear, analog-digital transducer). It digests the messages thus constituted by passing them through a filter program which is a “mechanical table” of values gauging, for example, the redundancy rate, the number of repetitions, the number of symmetry elements (Birkhoff). It analyses, from an informational point of view, the architecture of what it sees and of the surrounding sound world, as well as the hierarchy of their sequencing. It assigns an overall value resulting from the composition of the partial values to the different levels, according to rules dictated by the aesthetician who wrote the program, according to what he knows about the psychology of sensations. The machine then selects spectacles that exceed a certain value, qualifies them as “works of art”, stores them in a memory. It will be able to regurgitate them on demand by translating them at the other end into sensory phenomena, using, this time, a digital-analog transducer, a television receiver or a complex sound generator.

The machine helps the esthetitican in his role of mechanical critic. If the world is full of beautiful things, the critic will turn into an artist the moment?? he puts a frame around such a piece of macadam that his infallible gaze will have appreciated as aesthetic. It will be noted that in this case there is no “responsible” person. The program is provided by the pooled consensus of all humanity, and the source is the vast world.

Second attitude

Critical aesthetics: the machine spectator or artificial listener explores the world to highlight orderly relationships and imperceptible forms in human time.

This analysis performs an integral of the phenomena and spectacles of the external world at a level higher than that of which the human being is capable. We know that the human observer is, for the most part, unable to control the flow of originality that assails him, to establish a long-range order upon it; we know that his simplest reaction is renunciation.

A series of visual messages (t1, t2, t3…tn) are presented to an observer,and the “seeing machine” (in practice, a digital-analogue converter) transforms the visual forms into a deck of cards. It then carries out calculation operations and systematically analyzes the autocorrelation of the messages, possibly detecting structures of large-scale ordering between disparate picture elements. It provides a kind of phosphorescence of objective impressions, likely to bring out supersigns or shapes.

For example, in space, the phenomenon of moiré that is familiar to us is a supersign: this is the interference between series of superimposed regular elements. Back in the day, we came to know that stroboscopy or accelerated cinema sometimes construct visual forms that had remained impossible to ascertain in sequential observation.

Now, precisely, the limits of the field of consciousness of the human observer prevent him from “telescoping” such patterns if they are too distant or too slow, and therefore from perceiving these hidden correlations which may constitute definite forms. The expanded field of consciousness of the computer’s ferrite memories will supplement it. It will record these forms; sometimes phantom forms, sometimes new forms. It will regurgitate them on request, as a source of inspiration or as an object to see, that is to say a new spectacle.

Third attitude

Applied aesthetics: according to the method of cybernetic reduction, the machine spectator or artificial listener analyzes the cultural world, identifies analogical models that it makes operative in a simulation of creative processes.

The organization of the process is divided into two parts, corresponding to two machines or to two successive uses of the same machine programmed differently.

The analysis is oriented towards further experimentation. It will seek to trace the composer’s attempts and mistakes to possibly make a critique and grasp where their work transcends what a mechanical model is capable of accomplishing.

It is the “neo-Cartesianism” of the “imaginary machine” clearly stated by Philippot and put into practice by Hiller in his famous work that gave rise to the “Iliac Suite”.

In the analytical part, we find a translator of the sensory phenomena of the external world, but which applies here to the cultural world, namely, to the source of works of art recognized as such by common consent in the particular field in which we are interested: sound, vision, shape, color, etc. The converter translates into figures, by a statistical characterization, the objective traits of works of art of the past. It thus defines, as in paragraph 1, the symbols representing the sensory elements, and constitutes an ordered repertoire. To do this, it integrates by searching for laws of autocorelation; in other words, it generalizes.

At the end of this integration, the works are recorded in two separate memories — as long as the analysis has been well done — by their own rules, the rules by their multiple applications??. The first memory materializes the particular combinations of rules that express each particular work. The second lists multiple applications according to their common rules. Attentive to the world of art, respectful of what humanity has deemed most beautiful, the machine is anxious to condense its objective characters and store them.

In the synthetic part, the artificial spectator or artificial listener transforms himself into an artificial creator, uses the secrets of the Beautiful that he has measured, strives to reproduce the processes of creation that he masters. A source of imagination or randomness takes from the repertoire, duly defined beforehand, a first symbol. Then, each new symbol is subjected to a sequential analysis to determine whether it complies with the set of rules which have been released in the analytic part (first memory) and which, at this stage of the machine, are reinserted to carry out a simulacrum of a style, which may be the “Cantus Firmus” proposed by Fux, the symphony proposed by Beethoven, or the geometric abstraction proposed by Vasarely.

Or the particular symbol in question obeys all the rules determined for the experiment, in which case it will be transmitted and recorded in succession. Or he does not obey it, in which case he is rejected, and the random source is ordered to propose a new sign which will in turn be examined according to the same criteria, and then accepted or rejected. This iterative process starts again until a valid sign is found.

Thus, little by little, a sequence is established which follows exactly the tentative creative advances, the repentance and the turning back, to the aesthetic satisfaction within the framework of a given style. Up to this point, it can be said that this flowchart is that of the student of the conservatory trained in the abstract rules of composition and performing a counterpoint exercise by erasing the notes on his paper until a result is obtained that satisfies all the rules of the game.

The second stage of this synthesis will be a formalization of the whole; the results will then be compared with the repertoire of works that the second memory of the artificial spectator had recorded. The criterion of comparison is no longer conformity to a style, but originality; it will determine whether the work is valid, that is to say new. If it does not exist, it is possibly recorded or decoded, that is to say retranscribed in the “sensory language”, from the results in “machine language” by means of a digital-analog converter. It is then delivered for consumption.

It should be noted that, in this process, the aesthetician takes the role of an artist; after having defined the rules of Beautiful in the whole of art, he recreates other works according to the same rules, but he refuses to take responsibility for them. It is the machine that is responsible. The choice of the rules of a style having been defined, a random process is set in motion, in which the beautician contemplates the result.

This process materializes an important question: Did Brahms write all the “Brahms” he could write? And if Brahms doesn’t suit us, Tchaikovsky is a much better example. In the reinsertion of the set of rules that define the style of Brahms or Tchaikovsky and in the exploration of all possible works obeying exactly the same criteria, the machine proposes to the philosopher of art two equally disturbing answers.

If all the possible variations around the First Piano Concerto turn out to be a pale reflection of the original, a worthless marked-down version, a bad plagiarism, it is because it was an inaccessible ideal: there are other rules, more hidden and thus the analysis was insufficient and it must be repeated. This is a response of considerable interest to the art critic.

But if, by chance, the “remakes” were as good, or even better, than the original, it is because in exploring their field of possibilities, Tchaikovsky or Brahms had not chosen the preferred route in the path with multiple branches. There existed virtually a “First Concerto” better than the one that Tchaikovsky materialized in his writing: we would be very wrong to neglect this precious resource of artistic wealth. A huge field opens up for experimentation on models and applied aesthetics.

Fourth attitude

Abstract creation: the machine, an amplifier of complexity, develops an idea of composition.

The human mind is too weak for the ideas it imagines, it needs technical help: it is the computer that will provide it (Barbaud). An idea having been found by a responsible artist, he feels unable to bring it to completion, because the developmental work it demands exceeds human forces. This is the case of K.O. Goetz, who wants to explore the combinations of black and white signs to create “supersigns” and uses a team, each of whom — according to rules rigorously provided by the creator — makes a small piece of the overall picture. The work to be done rapidly exceeds the limits of human action, even organized in teams. It is to the machine that we will ask to serve as an amplifier of complexity, to push our desires to the limit and to realize them. We put an idea and a repertoire of symbPermutational art: the machine systematically explores a field of possibilities defined by an algorithm.ols in the machine and we ask it to develop this idea.

Xenakis sees an interest in distributing sound elements according to a number of simple rules and wonders what will happen to these rules as regards their perception by the listener. A priori, he did not know; he began by assembling grains of sound by hand, then, when the magnitude of the calculations exceeded him, he asked the I.B.M. 704 to execute them for him. He can then analyze, choose among the results that respect the laws of logical thought, those that are more properly aesthetic.

It seems that a good part of our future art can develop along these lines. But it takes stubborn rigour to bring this project to completion, since it is a question of always applying the same rule a very large number of times. Here the human will wears out quickly. It is time to make way for the machine and ask it to do the work. Until now, however, man had never had this partner of intelligence and work at his disposal, and his experiments in this field had remained extremely limited. So there is something new; a way has opened where art can rush in.

Fifth attitude

Permutational art: the machine systematically explores a field of possibilities defined by an algorithm.

The machine no longer has recourse to the field of natural possibilities (para. 1), the elements and rules of style of existing works (para. 3). It no longer follows all the implications of an abstract idea (para. 4), but it realizes all the possible works that the creator provided the program with — repertoire and idea — it composes with him. It comprises a symbolic code of sound or visual elements, constituting the repertoire; a field of possible combinations is then delimited by stating a “sequence of rules for the processing and arrangement of these elements”: this is the definition of a combinatorial algorithm. This field of possibilities is extremely wide; a human being could trace a particular path and innocently pass by the most attractive achievements.

Only the machine will be able to systematically apply the combinatorial game to all the elements of the algorithm, to explore and exhaust the whole of this field of possibilities. It will create a very large number, finite but immense, of potential works that it can keep in stock. But it is wiser to then screen - through a certain number of a priori values (intelligibility, sensuality, etc.) - each of the millions of works produced, to keep the best ones. What will remain on the sieve will be stored and then sold: it is the equivalent of a sociological survey on the pleasure of beauty. The ethetician becomes an artist as he creates his algorithm and is responsible for it. It will also establish the “title program” as in para. 1 1.

This is the attempt of the algorithmic music of Barbaud; it is the S+7 method illustrated by Lescure at the Oulipo; it is the method of variations of Kuhlmann or Picard etc. his Permutational Art, delicious game of mandarins who execute it, has considerable importance in a consumer society to which it brings personal diversity in the uniformity of the same algorithm. Each Prisunic customer will have a unique and irreplaceable pattern of personalized marquetry on his or her tabletop in formica, supplied for himself or herself by a machine artist that is able to churn out millions of others at the cost of the very same program. This is the idea of the multiple.