|
|
|
LAD Conference |
|
ZKM Karlsruhe |
|
14-16 March 2003 |
|
|
|
Frank van de Pol |
|
|
|
|
|
|
Purpose of this talk: |
|
|
|
Terminology |
|
Synchronisation concepts |
|
Awareness of difficulties |
|
New Ideas? |
|
|
|
|
|
|
|
|
|
|
Time Reference |
|
Deadlines & Delay management |
|
Soft Synth considerations |
|
Synchronisation |
|
Questions |
|
|
|
|
|
|
|
|
|
In music it is important to agree upon the
notion of time |
|
Notes and other events need to be played at a
certain time |
|
this time stamping is important for recording
too |
|
Different unit of measure depending on
application |
|
|
|
|
|
|
|
Wall clock, gettimeofday() |
|
Relative time |
|
Seconds since specific point. |
|
This can also be sample count too |
|
commonly used in audio recorders |
|
Media Time – position [s] in song |
|
Musical Time |
|
Bar/Beats/Tempo |
|
commonly used in sequencers/MIDI |
|
|
|
|
|
|
|
|
|
|
Delays are inevitable in any audio system
(especially digital) |
|
A/D & D/A conversion |
|
Buffers (operating system, drivers, application, |
|
Block/frame processing |
|
Software, PCI buss dma transfers |
|
Processing time for algorithms |
|
Outboard gear |
|
Natural acoustic delay |
|
|
|
|
|
The velocity of sound in air is approximate 343
m/s (at 20°C) |
|
|
|
Adding a delay of 1 ms is comparable to moving
your speakers 34 cm further away
(or 13.5” for our non-metric
friends) |
|
|
|
10 m corresponds to 29 ms delay |
|
|
|
|
|
Small constant delay is never a problem |
|
Headphones instead of speakers at 3m distance
saves you 9ms. |
|
|
|
Relative low latency desired for feedback to
musician or engineer |
|
How much is good enough? |
|
Latency for pneumatic organ players |
|
|
|
High Latency peaks are very Bad!! |
|
|
|
|
|
Variable delay results in poor stereo imaging |
|
Human brain uses difference in arrival time |
|
|
|
Variable delay ruins subtle musical timing |
|
Easily noticed |
|
The ‘groove’ in music (eg. Drums & bass
guitar) |
|
|
|
|
|
Our Biggest problem |
|
Characteristics as Jitter but systems are far
more vulnerable for it |
|
Amount of delay is easily substantial |
|
|
|
Solving it using Low Latency? |
|
Makes the effects less dramatic |
|
Good workaround to keep the amount of delay down |
|
Killing a mosquito with a cannon? |
|
|
|
|
|
|
|
|
|
|
|
Low Latency is great but don’t exaggerate |
|
|
|
Minimize the variability in delay |
|
|
|
Know your delay along the path: Align media
streams by adding additional delay. Compensate for the slowest |
|
Calibrate if delay is unknown |
|
|
|
|
|
|
Many Soft synths available using ALSA Sequencer
API |
|
|
|
Events delivered to Synth at scheduled time |
|
|
|
Low Latency processing to get the output
delivered shortly after event reception |
|
|
|
|
|
|
|
It works! |
|
|
|
Hardware MIDI equipment uses the same approach. |
|
|
|
Delay is unknown and not always constant /
predictable |
|
Event is processed as fast as possible |
|
|
|
No Sample Accurate sync possible |
|
|
|
|
Use Call Back API for audio data |
|
|
|
Receive Media Time for every frame to compute |
|
|
|
Receive events using ALSA Sequencer or similar
in advance |
|
|
|
Handle micro scheduling in the algorithm |
|
|
|
|
|
|
|
Sample accurate scheduling |
|
|
|
No worries about latency |
|
Just use constant delay |
|
|
|
Any real-time to media time ratio possible |
|
Faster than real-time rendering |
|
Offline rendering for cpu time intensive
algorithms – eg csound |
|
|
|
|
|
Call back interface with Media Time required |
|
JACK is only real-time |
|
MUSE instrument API |
|
XAP is still vapor |
|
OS-X CoreAudio Audio Units |
|
|
|
ALSA Sequencer does not deliver unscheduled
events (yet) |
|
Needs to be added for migration to user space
ALSA-Lib too! |
|
|
|
|
|
|
|
|
MIDI to Audio in DAW |
|
Drum machines, sequencers, arpeggiators, tempo
delays |
|
Audio to Video |
|
Audio to Audio |
|
Tape decks (reel to reel) |
|
Other external gear |
|
|
|
Many other things…. |
|
|
|
|
|
Media Time (eg. SMPTE code) |
|
Digital Audio |
|
Video |
|
Musical Time (eg. MIDI clock) |
|
Sequencers |
|
|
|
Conversion needed |
|
software, special hardware |
|
Relation Musical Time to Media Time |
|
Relation Media Time to System Time |
|
|
|
|
|
|
|
|
|
Synchronised clocks available: |
|
(eg. Word-clock) |
|
Distribution and alignment of Media Time still
needed. |
|
Position |
|
|
|
Word Clock is fast & sample accurate, but
what about clocks with high granularity (video frames)? |
|
|
|
Derive your clock… |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Align according to reference |
|
Unit of measure: time, beats, frames… |
|
|
|
|
|
2nd stream is slower (m>n) |
|
<Unit>/t ratio for 2nd stream =
m/n |
|
|
|
|
|
Difficulty is to perform accurate measurement of
the ‘anchor points’… Compare to what? |
|
Accurate time source needed
(eg. cpu cycle counter) |
|
|
|
Risk of introducing too much jitter |
|
requires appropriate filtering |
|
|
|
Interpolation needed |
|
|
|
|
|
For time stamping and measurements an accurate
clock with very fine granularity is needed. |
|
Eg. Pentium CPU Cycle Counter |
|
|
|
For scheduling an interrupt source is needed. |
|
RTC, new (v2.6) HighRes APIC timers and posix
timers |
|
|
|
|
|
|
The ability to keep streams in sync is not
enough |
|
|
|
Changes to media time should be distributed too: |
|
Transport controls needed |
|
|
|
Follow SMPTE start/stop/cue model? |
|
|
|
|
The Virtual Studio Concept seems to become or is
already reality at the “Big Commercial Guys”. |
|
|
|
We need consider how we glue our applications
and tools together to make them cooperate. |
|
|
|
|
|
|
|
|
|
|
|
Notes
