This is a heavily abridged version of the Reactive Backing paper presented at the Innovation in Music conference at Anglia Ruskin University, Cambridge, England in 2015. The citation for the full paper is as follows:

Canfer, T. (2015). A System of Reactive Backing for Live Popular Music. KES Transactions on Innovation in Music: Vol 2, 26-35.

A System of Reactive Backing for Live Popular Music

Tim Canfer

Introduction

Current trends for the use of technology as accompaniment in Popular Music are still very much stuck in the pre-recorded backing track tape technology of the 1960s and 1970s, requiring the musician(s) to slavishly follow the click track to stay in sync. Playing at the same tempo with the same music for every performance takes much of the life out of it and is a restriction that has not been widely addressed in the field of Music Technology. In Art Music there are many technological systems for augmenting accompaniment, but these are generally systems of Score Following, which are inherently unsuitable for the more flexible requirements of Popular Music.

The terms Art Music and Popular Music, used within this article, are broadly based on the distinctions offered by Tagg.[1] That is, Popular Music, including such genres as Rock and Pop, is music generally created for mass production, explicitly as a commodity. Popular Music is generally not stored or distributed in written form and does not have a history of theoretical and aesthetic analysis. This is in contrast to Art Music, including Classical Music and Electroacoustic Music, which is music that generally does have a history of theoretical and aesthetic analysis, is commonly stored or distributed in written form, but is not created explicitly for mass production as a commodity.

This work presents a novel solution to restrictive backing tracks. It aims to demonstrate new ways in which a laptop computer can provide a system of flexible and effective Reactive Backing to enhance performance and to explore the feasibility of using live foot tapping data in contrast to live audio to drive a tempo control device.

Background and Related Work

There is a wide range of automatic accompaniment solutions available for Art Music. These mainly employ the technique of Score Following, that is, anticipating known musical events and following them using a variety of note detection methods. Current commercial Score Following programmes include: Tonara, SampleSumo, and Cadenza. Current academic systems include IRCAM’s Antescofo~[2] and the IMuSE[3] system.

In Popular Music however, a score is not generally used. A musician playing Popular Music will usually be expected to play from memory and by ear and have a far greater degree of flexibility than a musician playing Classical Music. Because of this, a system of Reactive Backing for Popular Music must try to track the beat of the music without the reference of a score. This is done using the technique of Beat Tracking. It is because there is no knowledge of when the notes occur that this is inherently a far more difficult technique with which to drive a reliable system of Reactive Backing. 

It is worth noting that there is also the very widely implemented and simple process of Tap Tempo, which calculates the tempo based on the time between successive taps. Tap Tempo is included as a part of live Popular Music applications such as Ableton Live and Mainstage and is able to control the tempo of these applications in real time. Tap Tempo’s main use is as a rough tempo identification tool. As a live tempo varying tool, it suffers from several significant drawbacks:

1.     The tempo initially changes in large instantaneous jumps.

2.     Every beat must be tapped for it to work. 

3.     The tempo generally gets progressively averaged, making the system less and less responsive as you continue to use it.

Discussion of Foot Tapping Data

The main focus of this work is to examine the feasibility of foot tapping data (instead of analysed audio), to drive a device that varies the tempo of a backing track in real time. The three main reasons for using foot tapping are discussed below:

1.     The tactus (or beat) of music is defined concisely by Davies and Plumbley as ‘the rate at which humans are most likely to tap’ (p. 1010).[7] The idea of using foot tapping information is to cut out the audio analysis and go straight to the musician’s own physical interpretation of the beat.

2.     Foot tapping to music is widely encouraged in Popular Music teaching to ensure a more consistent performance. (In comparison, foot tapping is widely discouraged in Classical Music. While this contrast is interesting, its discussion is outside the scope of this work.)

3.     There is an inherent difficulty in achieving a Beat Tracker reliable enough for a live system of accompaniment (especially compared with Score Following). This is compounded when considering the audio signal of a guitar and vocal, which is particularly rhythmically ambiguous in comparison with a drum kit. 

The process of tapping your feet along with music is a typical example of what is known in Psychonomics as Sensorimotor Synchronisation (SMS), defined by Repp[8] as ‘the rhythmic coordination of perception and action.’ (p. 969) There are many studies of SMS and they give an idea of how useful or reliable foot tapping may be as a means to drive a system of Reactive Backing.

One of the oldest and most consistently backed-up findings in SMS research is the anticipation tendency, or the mean negative asynchrony (MNA). As Repp points out, this is the evidence that in tapping in time with an auditory metronome the taps tend to precede the tones by a few tens of milliseconds.[9] 

In an early experiment published initially in 1971, Fraisse and Voillaume demonstrated that if the metronome sound is switched mid-experiment to be generated by the tapping itself by an unsuspecting listener, the tempo steadily accelerates. In their conclusion, regarding the subjects of the experiment who were unaware of the change in tap generation, they found that the subjects didn’t like to tap with the beat, they preferred to tap ahead of the beat.[10]

With regard to the synchronisation to music by musicians there are two relevant studies. Firstly, Aschersleben’s evidence that the MNA of the trained musicians she tested was 14ms compared with 40 to 50ms for non-musicians.[11] Secondly, and most importantly, the work of Wohlschläger and Koch which showed that the MNA tends to disappear when additional tones or movements are inserted in between metronome tones or the taps. They go on to suggest that the MNA might only occur under artificial laboratory conditions.[12]

Clearly there is a wealth of tones and movements in between the beats when playing music live. These experiments and summaries strongly support foot tapping as a method of driving a system of tempo control, particularly with trained musicians. It is also noted that an offset function to allow for consistent asynchrony would be desirable for this system.

Relative Tap Tempo

The tempo control element of this system of Reactive Backing is called Relative Tap Tempo. This is similar to Tap Tempo however, Relative Tap Tempo works by comparing the position of a foot tap to the existing beat of the backing track while it plays, rather than simply counting the time in between taps.

This is shown graphically below in Figure 1. The foot taps are represented as diamond shapes on a timeline of one bar of music split into four beats. A new tempo is calculated for each foot tap within the deviation window range (set by the Dev % control), using the time interval from the last beat that is more than half a beat away from the foot tap. If the foot tap occurs before the beat (foot tap a), the new beat interval is used to generate a faster tempo. If the foot tap occurs after the beat (foot tap b), the new beat interval is used to generate a slower tempo. The new tempo is updated immediately and the amount of change can be scaled down by the sensitivity setting (set by the Sens control) and smoothed by the moving average setting (set by the Ave control), see controls below Figure 3. Any foot tap outside the deviation window range is ignored (foot tap c).

This design works towards the ideal of the computer acting as another musician to be interacted with.

In the initial stages of development, a footswitch was used to register the foot taps. However, it became clear that this interface was a significant restriction to performance, so a tapping sensor using an accelerometer from a Wii remote nunchuk gaming controller strapped to the musician’s shoe was developed (see Figure 2). The accelerometer, rather than a foot switch, allows for much more freedom of movement and the tapping is far more natural.

Practical Use

Use of the Reactive Backing system has proved encouraging. The system is designed specifically for the use of the author and reacts as hoped, although there was a significant period of training to get used to using the system smoothly. Several successful live performances have demonstrated the suitability of the system to a live Popular Music environment and it will be particularly interesting to test the system with other musicians.

An interesting and unexpected use of the Relative Tap Tempo (Wii_OSC) device is for real-time performance timing analysis in practice. The panel behind the tempo display flashes a colour dependent on how much of a timing change is triggered by each foot tap. The colour goes from bright green at the limit of an early tap, to yellow on the beat and then to red at the limit of a late tap. Using this, the musician can see immediately how they are deviating from the tempo and can use this to correct unwanted tempo fluctuations in performance.

Conclusion

This system of Reactive Backing bypasses the traditional method of interpreting the tempo from either Audio or MIDI information from played music in favour of Foot Tapping data; a method that aims to provide both a more direct and discrete alternative means to control tempo. Previous work in the related field of Synchronisation shows a solid theoretical basis for this approach. Also, the development of Dannenberg’s basic foot tapping system, as well as the author’s use of the Relative Tap Tempo (Wii_OSC), Dynamic Control and arrangement devices is highly encouraging.

The author’s use of the system so far demonstrates that while this approach works, but that it requires a significant amount of rehearsal, and a compromise needs to be found to balance a fast system response against an unpleasant tempo slew.

Future plans are to test the system thoroughly with other musicians and to continue refining and developing the devices.

References 

[1]   Tagg, P. Analysing Popular Music: Theory, Method and Practice, Popular Music, Vol. 2, pp. 37–67, (1982)

[2]   Cont, A. ANTESCOFO: Anticipatory Synchronization and Control of Interactive Parameters in Computer Music, Proceedings of the International Computer Music Conference (2008)

[3]   Ritter, M., Hamel, K and Pritchard, B. Integrated multimodal score-following environment, Proceedings of the International Computer Music Conference (2013)

[4]   Ingo, V. Circular Logic InTime, Real-time Tempo Tracking Software [Mac/PC]. Accessed March 2015 from http://www.soundonsound.com/sos/jun05/articles/ intime.htm

[5]   Robertson, A. and Plumbley, M. D. Synchronizing Sequencing Software to a Live Drummer, Comput. Music J., Vol. 37, No. 2, pp. 46–60, (2013)

[6]   Dannenberg, R. B., Gold, N. E., Liang, D. and Xia, G. Methods and Prospects for Human–Computer Performance of Popular Music. Comput. Music J., Vol. 38, No. 2, pp. 36–50, (2014)

[7]   Davies, M. E and Plumbley, M.D. Context-Dependent Beat Tracking of Musical Audio, IEEE Trans. Audio, Speech, and Language Process., Vol. 15, No. 3, pp. 1009–1020, (2007)

[8]   Repp, B. H. Sensorimotor synchronization: A review of the tapping literature, Psychon. Bull. Rev. Vol. 12, No. 6, pp. 969-992, (2005)

[9]   Repp, B. H. Musical synchronization (Altenmüller, E., Wiesendanger, M and Kesselring, J. Editors) Music, Motor Control and the Brain. Oxford University Press, pp. 55–76, (2006)

[10]Fraisse, P., Voillaume, C and Repp, B. H. The subject’s references in synchronization and pseudo-synchronization, Psychomusicology: Music, Mind and Brain, Vol. 20, No. 1-2, pp. 170-176, (2009)

[11]Aschersleben, G. Temporal Control of Movements in Sensorimotor Synchronization, Brain and Cognition, Vol. 48, No. 1, pp. 66–79, (2002)

[12]Wohlschläger, A and Koch, R. Synchronization error: An error in time perception (Desain, P and Windsor, L. Editors) Rhythm perception and production. Swets, Lisse, pp. 115–127, (2000)