Display | FORMS

FORMS is a generative visual-music scoring bot

It can self-generate infinite graphic scores driven by a set of rules based on probability and chance. Scores are transformed automatically into music using sound synthesis algorithms, blurring the lines between partiture and spectrogram.

The resulting images are framed in the tradition of graphic notation popular with 20th-century composers including John Cage, Iannis Xenakis, Karlheinz Stockhausen and Morton Feldman as a means of representation that released music from the rigid bounds of conventional pentagrams.

FORMS has been “playing” online uninterrupted for more than 2,000 hours, broadcasting its mesmerizing musical-visual output at twitch.

Following a well-deserved rest, and for the exhibition at Display, FORMS is back, this time as a group of instruments. A series of screens interpret one instrument each. Thanks to its networked brain, these instruments can coordinate with each other, each one playing its part in a real-time musical composition never to be repeated.

Graphic notation

FORMS is heir to the advances made in graphic musical notation as of the second half of the 20th century.

Find out more

Composers such as Iannis Xenakis, John Cage, Mestres Quadreny, György Ligeti and Karlheinz Stockhausen opened up a whole spectrum of new possibilities in music composition, expanding into new territories and liberating music from the rigid bounds of traditional notation.

Xenakis's Metastaseis, Ligeti's Artikulation and a large part of the early production of Morton Feldman are direct references in the works generated with FORMS.

Image 2 of the gallery "<p>Composers such as Iannis Xenakis, John Cage, Mestres Quadreny, György Ligeti and Karlheinz Stockhausen opened up a whole spectrum of new possibilities in music composition, expanding into new territories and liberating music from the rigid bounds of traditional notation.</p>
<p>Xenakis's Metastaseis, Ligeti's Artikulation and a large part of the early production of Morton Feldman are direct references in the works generated with FORMS.</p>"

Image 3 of the gallery "<p>Composers such as Iannis Xenakis, John Cage, Mestres Quadreny, György Ligeti and Karlheinz Stockhausen opened up a whole spectrum of new possibilities in music composition, expanding into new territories and liberating music from the rigid bounds of traditional notation.</p>
<p>Xenakis's Metastaseis, Ligeti's Artikulation and a large part of the early production of Morton Feldman are direct references in the works generated with FORMS.</p>"

Image 4 of the gallery "<p>Composers such as Iannis Xenakis, John Cage, Mestres Quadreny, György Ligeti and Karlheinz Stockhausen opened up a whole spectrum of new possibilities in music composition, expanding into new territories and liberating music from the rigid bounds of traditional notation.</p>
<p>Xenakis's Metastaseis, Ligeti's Artikulation and a large part of the early production of Morton Feldman are direct references in the works generated with FORMS.</p>"

Image 5 of the gallery "<p>Composers such as Iannis Xenakis, John Cage, Mestres Quadreny, György Ligeti and Karlheinz Stockhausen opened up a whole spectrum of new possibilities in music composition, expanding into new territories and liberating music from the rigid bounds of traditional notation.</p>
<p>Xenakis's Metastaseis, Ligeti's Artikulation and a large part of the early production of Morton Feldman are direct references in the works generated with FORMS.</p>"

Image 6 of the gallery "<p>Composers such as Iannis Xenakis, John Cage, Mestres Quadreny, György Ligeti and Karlheinz Stockhausen opened up a whole spectrum of new possibilities in music composition, expanding into new territories and liberating music from the rigid bounds of traditional notation.</p>
<p>Xenakis's Metastaseis, Ligeti's Artikulation and a large part of the early production of Morton Feldman are direct references in the works generated with FORMS.</p>"

Image 7 of the gallery "<p>Composers such as Iannis Xenakis, John Cage, Mestres Quadreny, György Ligeti and Karlheinz Stockhausen opened up a whole spectrum of new possibilities in music composition, expanding into new territories and liberating music from the rigid bounds of traditional notation.</p>
<p>Xenakis's Metastaseis, Ligeti's Artikulation and a large part of the early production of Morton Feldman are direct references in the works generated with FORMS.</p>"

Image 8 of the gallery "<p>Composers such as Iannis Xenakis, John Cage, Mestres Quadreny, György Ligeti and Karlheinz Stockhausen opened up a whole spectrum of new possibilities in music composition, expanding into new territories and liberating music from the rigid bounds of traditional notation.</p>
<p>Xenakis's Metastaseis, Ligeti's Artikulation and a large part of the early production of Morton Feldman are direct references in the works generated with FORMS.</p>"

Hide

Spectral
sonification

The field of sound synthesis encompasses developments in the manipulation of spectrograms that link into FORMS research. Spectrograms are visual representations of the frequencies contained in a signal, whether sound or other waves on the electromagnetic spectrum.

Find out more

In the music world, spectrograms are commonly used as sound analysis tools offering an understanding of frequency content so that potential sounds can be repaired or as a visual reference supporting the listening experience.

FORMS subverts this relationship as it generates the spectrogram first and then transforms it into sound waves.

The software responsible for sonification reads the image at the point where the header is situated, assigning a frequency on the audible spectrum to each pixel in that column. Pixels located lower down on the image have a lower frequency, while those higher up are higher pitched. The brightness of each pixel in the column determines the volume of each frequency, so darker pixels sound at a lower volume and brighter pixels with a tone closer to white will sound at a higher volume.

Hide

Visual music and procedural art

The nature of FORMS research links in with fields of procedural art and visual music – both analogue and digital.

Find out more

Experimental filmmakers including Walter Ruttmann and Oskar Fischinger were pioneers in this visual approach to music in the early 20th century.

At the same time, visual artists such as Sol Lewitt, Vera Molnar and Julio LeParc began introducing procedural elements into their output: art was no longer the finished piece but a set of rules for artistic production. The process is what brings art about.

Image 2 of the gallery "<p>Experimental filmmakers including Walter Ruttmann and Oskar Fischinger were pioneers in this visual approach to music in the early 20th century.</p>
<p>At the same time, visual artists such as Sol Lewitt, Vera Molnar and Julio LeParc began introducing procedural elements into their output: art was no longer the finished piece but a set of rules for artistic production. The process is what brings art about.</p>"

Image 3 of the gallery "<p>Experimental filmmakers including Walter Ruttmann and Oskar Fischinger were pioneers in this visual approach to music in the early 20th century.</p>
<p>At the same time, visual artists such as Sol Lewitt, Vera Molnar and Julio LeParc began introducing procedural elements into their output: art was no longer the finished piece but a set of rules for artistic production. The process is what brings art about.</p>"

Image 4 of the gallery "<p>Experimental filmmakers including Walter Ruttmann and Oskar Fischinger were pioneers in this visual approach to music in the early 20th century.</p>
<p>At the same time, visual artists such as Sol Lewitt, Vera Molnar and Julio LeParc began introducing procedural elements into their output: art was no longer the finished piece but a set of rules for artistic production. The process is what brings art about.</p>"

Image 5 of the gallery "<p>Experimental filmmakers including Walter Ruttmann and Oskar Fischinger were pioneers in this visual approach to music in the early 20th century.</p>
<p>At the same time, visual artists such as Sol Lewitt, Vera Molnar and Julio LeParc began introducing procedural elements into their output: art was no longer the finished piece but a set of rules for artistic production. The process is what brings art about.</p>"

Image 6 of the gallery "<p>Experimental filmmakers including Walter Ruttmann and Oskar Fischinger were pioneers in this visual approach to music in the early 20th century.</p>
<p>At the same time, visual artists such as Sol Lewitt, Vera Molnar and Julio LeParc began introducing procedural elements into their output: art was no longer the finished piece but a set of rules for artistic production. The process is what brings art about.</p>"

Image 7 of the gallery "<p>Experimental filmmakers including Walter Ruttmann and Oskar Fischinger were pioneers in this visual approach to music in the early 20th century.</p>
<p>At the same time, visual artists such as Sol Lewitt, Vera Molnar and Julio LeParc began introducing procedural elements into their output: art was no longer the finished piece but a set of rules for artistic production. The process is what brings art about.</p>"

Image 8 of the gallery "<p>Experimental filmmakers including Walter Ruttmann and Oskar Fischinger were pioneers in this visual approach to music in the early 20th century.</p>
<p>At the same time, visual artists such as Sol Lewitt, Vera Molnar and Julio LeParc began introducing procedural elements into their output: art was no longer the finished piece but a set of rules for artistic production. The process is what brings art about.</p>"

Hide

A unified audiovisual language

FORMS integrates music and visual art to generate a single unified material. These are not symbolic scores in a secret language. Rather, the graphics literally are the music.

Find out more

Neither is the music created by FORMS a free representation stemming from visual inspiration. The music is literally the drawing. FORMS moves away from any romantic artistic ideals to offer a kind of audiovisual art based on a body of scientific knowledge that generates an objective representation of sound.

Nevertheless, efforts have been made to ensure that both the visual and sound output comply with certain minimum requirements for aesthetic quality in terms of the chromatic values, composition and drawing in the visuals and the timbre, tonality and rhythm of the sounds.

The essential quality of this project's concept makes it ideal for developing in multiple formats: FORMS is not a collection of paintings or a concerto for string quartet. It is not an immersive installation or software tool. It is a new audiovisual language, a new way of composing images and sound in a unified way to generate highly attractive visual-musical compositions. In a way, it is Kandinsky's dream come true: a pictorial-sound language that does not differentiate between seeing and listening, in which every geometric figure, every dot or line, can be interpreted by the eye and the ear alike. This new language can be applied to multiple formats including art collection, concerts, installation art or software without losing its core characteristic: its ability to bring together music and visual art in a single, compact material.

Hide

Algorithms and maths

FORMS stems from a whole series of developments in mathematics and computer programming.

Find out more

A mathematical matrix on the visual canvas is used to create tonal drawings and coherent rhythms. This grid establishes a set of horizontal divisions defining tonally relevant points and a set of vertical divisions defining rhythmically relevant points. For the purposes of creating visual compositions that sound good, it is necessary to establish a set of drawing rules linked to the position, spacing and scale of geometric figures in relation to the grid. To define these rules, mathematical algorithms are created that take into account the coordinates of each of the divisions on the grid, along with parameters such as tempo and tonality.

A set of algorithms based on chance and probability establish the prevalence of one type of visual figures/sounds over others, organizing them into “families” or “songs”. The resulting audiovisual composition will depend on the set of geometries that prevail from this play of chance and probability.

Hide

Actions on the work

Form string quartet

"FORMS - String Quartet" is a live multimedia performance for string quartet, electronic music and panoramic video projection, based on the concept of "graphic sonification".

Find out more

The musicians perform a series of graphic scores that in turn build the visual scenography, offering the audience the experience of being able to anticipate the "music to come".

These scores, together with the spectrograms that make up the electronic accompaniment of the piece, have been previously created using generative graphic algorithms.

Santi Vilanova is a graphic design graduate who is passionate about sound art. He combines these two disciplines through the catalyst of creative technologies as a self-taught developer.

Educated on the rave scene of the early 2000s, his sound works have evolved to bring this influence to bear when breaking new ground. His recent research combines digital algorithms or sound engines with traditional staves and acoustic setups with a focus on the concept of visual music.

Along with Eloi Maduell, he is founder of the audiovisual research study Playmodes, which develops projects that push the boundaries of design, art and science.

Playmodes crystallizes a series of interests related to acoustics, maths, the psychology of perception, and light phenomena. From opera staging to immersive installations, the eclectic projects generated by Playmodes always have a common thread: the creative use of digital tools and quest for a unified audio-visual language.

The collective's works have been exhibited at light festivals (Fête des Lumières, Signal, Llum BCN, Mapping Festival, LIT Bogotá, Chartres en Lumières); at electronic music events (Sónar, Mira, LEV, DGTL, D4N Houston); and at contemporary art and theatre institutions (ZKM, Macau Arts Festival, In Between Time, Copenhagen Opera). It has also received a range of international awards in the sphere of new media (Ars Electronica – Honorary Mention, LAUS, Visual Music awards, and LAMP awards).

Vilanova is currently a university professor at the EINA University School of Design and Art (Autonomous University of Barcelona), Open University of Catalonia (UOC) and the BAU Design College of Barcelona (University of Vic), where he gives classes to new generations of multimedia creatives.