Joshua Batty

Grain Reordering Patterns in Kortex

23-06-2017

Grain Reordering Patterns

The primary aesthetic quality of AVGS lies in the structuring and placement of audiovisual grains over time. Although different audiovisual content fed through AVGS will trigger a unique perceptual relationship to the work, of primary importance are the underlying patterns that define the temporal re- orderings of the audiovisual grains. In this way, the AVGS technique is not biased to specific content and encourages playful experimentation using a wide range of source material. I have experimented with source material ranging from computer-generated audiovisuals, pop star music videos clips, martial arts fight scenes to streaming live television channels.

The fascinating qualities that emerge using any of this material is the perceivably impossible forms and behaviours that can occur as a result of digitisation. For example, AVGS applied to a martial arts scene can produce an illusion of the attacker performing physically impossible strikes on an opponent. Additionally, AVGS applied to musical source material featuring a vocalist will facilitate the emergence of unnatural melodic patterns. Essentially, the effect of observing the digitally manipulated human form short-circuits our hardwired expectations on how the form should behave.

The above diagram shows a collection of strategies that may be employed to modulate and shift the perceived temporal structure. Each control strategy facilitates a unique perceptual relationship between the viewer and the resulting audiovisual aesthetic. Within each pattern, the user can control the speed at which the pattern is read through, the direction of the play head through the pattern, and the pattern’s amplitude. Subtle manipulations of these variables during a performance can have a dramatic effect on how the audiovisual construct is perceived.

The grain reordering patterns presented in the above image help the viewer to experience the illusion of seeing multiple points in time simultaneously via harnessing persistence of vision. Although the shapes re-order both the auditory and visual grains, the brain perceives the auditory and visual stimulus in different ways. As the rate of reordering increases from 20 to 60 times a second, the auditory grains begin to fuse into a more continuous tone. Likewise, as the rate increases the brain perceives a greater overlapping visual montage, as a result of the visual short-term memory phenomena. Primarily, the retinal afterimages that AVGS produces facilitate the perceived breakdown of logical time and assist the performer to create a sensory overload environment for the audience.

Grain Reordering Patterns:

  • Loop Mode Forward (Ramp) = Machinegun-like effects (very rhythmical)
  • Loop Mode Reverse (Saw) = Time reversed machinegun-like effects
  • Loop Mode Palindrome (Triangle) = Similar to the previous mode except the temporal momentum of forward moving time is blurred
  • Shift Pause (Square) = Time freezing. Play head jumps to positive and negative affecting the audiovisual grain to be frozen
  • Random (S&H) = No perceivable pattern to the re-ordering
  • Exotic (Additive) = Blending two oscillators. Manipulating the three oscillators’ amplitude and frequency parameters can create a near infinite amount of patterns