CICA AVS4 SONIFICATION MODULES Please note that these modules are in Beta release. Please report any bugs to the address at the bottom of this file. These modules facilitate data sonification and sound synthesis by allowing modules to send sound synthesis commands to the Sonify module. Brief descriptions of the modules follow. Full documentation for each module is available. Sonify The Sonify module will synthesize sounds and can be controlled by other modules via command string input. It is most useful for data sonification where a data-producing module will send commands to Sonify based on the data represented. GraphSound The GraphSound module generates a graph of the waveform of a sound produced by the Sonify module and is useful as a diagnostic tool. All modules run as-is only under AVS4. They would require some modification to run under AVS3 or earlier. The Sonify module will only run as-is on Silicon Graphics platforms that support sound. The modules and documentation are available via anonymous ftp from ftp.cica.indiana.edu (129.79.20.84) in the directory pub/avs/Sonify. This software is free but we ask that: 1) You fill out the registration form included in the file "REGISTRATION". 2) You let us know of any enhancements you make to the software, and of any bugs you find in the software. 3) If you use the software in any form that you acknowledge its origin. 4) No charge is demanded for software containing any part of this code. See the file "COPYRIGHT" for full copyright information. We are interested also in any comments you might have or features and enhancements you would like to see in future releases. We are interested in keeping track of who is using the modules so that we can inform you of future releases, enhancements, and modifications. Please mail the registration form to: kaplan@cica.indiana.edu Brian Kaplan Center for Innovative Computer Applications Indiana University Poplars 808 Bloomington, IN 47405 812-855-6973 AVS Modules Record Animation(6) NAME Sonify - synthesize module-controllable sounds for data sonification SUMMARY Name Sonify Type Data Input Input string (optional) -- command string Output field uniform float -- sound field Params Name Type Description output sound toggle output sound from sound output port mode radio put module into marker/mapper mode time slider set the total time for output sound channels radio put module in mono/stereo mode frequency radio select the frequency of output sound sounds integer number of sounds in the output sound current sound islider select the sound to edit name string the name of the current sound switch radio whether the current sound is on or off waveform choice the waveform of the current sound sample file browser select an .aiff file for sampled sound speaker radio which speaker the sound will play on pitch (freq) slider the pitch of the current sound volume (amp) slider the maximum volume of the current sound attack rate slider the rate of attack of the current sound decay rate slider the rate of decay of the current sound sustain level slider the level of sustain of the sound release rate slider the rate of release of the current sound start clip slider start of the clip window of the sound end clip slider end of the clip window of the sound start place slider start of the placement window of sound end place slider end of the placement window of sound repeat radio turn on/off waveform repeating A slider parameter for waveform of current sound B slider parameter for waveform of current sound C slider parameter for waveform of current sound D slider parameter for waveform of current sound E slider parameter for waveform of current sound sound file browser select sound settings file to load/save load file oneshot load a sound settings file save file oneshot save a sound settings file DESCRIPTION The Sonify module is a coroutine module used for synthesis of sounds. It can be controlled either interactively, or by sending commands from other modules via the text string input. This allows the module to perform data sonification by allowing another module to change the sound parameters as it cycles through data. The Sonify module can work in two different modes: marker and mapper. Marker mode is useful for marking certain events that may occur in the data by playing a specified sound. In mapper mode, a tone is played continuously, and can be changed at any time by the controlling module. When in marker mode, the Sonify module will execute synchronously with the network (like a subroutine) so that the sound will finish playing before control is returned to the network. In mapper mode, the module executes asynchronously and the sound is played continuously, allowing the controlling network to change the sound as needed. General Settings Menu The module controls are divided up into two main menus: General Settings and Sound Settings. The general settings affect the output sound, which is an addition of all of the individual sounds created by the module. The marker/mapper selector allows you to select the mode that Sonify will run in. The time slider will select the length of the output time in seconds (the default is .5 seconds). When in marker mode, the time should be long enough to play the sounds you are generating. When in mapper mode, it is sometimes useful to set this to a relatively small number since changes can only affect the sound in between sound segments as the output sound is repeated continuously. The mono/stereo selector is used to determine whether the output sound is in mono (all component sounds are played through both speakers) or stereo (some components may be played on the left, some on the right, and some on both speakers). The frequency selector allows you to choose the frequency (in cycles per second) of the output sound. The "none" option turns off the output sound. The Sonify module will not produce any output until a frequency has been chosen. Higher frequencies increase the resolution of the resulting sound (although in most cases, a frequency of 8000 cycles per second is adequate). The output sound button determines whether or not Sonify will build a field on its sound output port. This field is useful for other modules which can graph or otherwise modify the sound (see OUTPUTS below). Sound Settings Menu The sound settings affect the individual sound components which are summed to create the output sound. The number of sound components is controlled by the "sounds" typein. The "current sound" slider allows you to select which sound is being edited. The other sound parameters will change to reflect the current sound. The "on/off" selector turns on or off the current sound controlling whether or not it is summed with the other sounds to produce the output sound. The "speaker" selector allows you to choose whether the sound is played on the left, right, or both speakers if the Sonify module is in "stereo" mode (see "General Settings Menu" above). The "speaker" selector has no affect if the module is in "mono" mode. The remaining "sound settings" parameters are divided into three submenus: "set waveform", "set envelope", and "load/save sound". The sound's waveform is it's base sound, and it's envelope defines how the volume of the sound changes as a note is played. The sound's clipping and repeating parameters are also in the "set envelope" menu. Loading and saving sound settings into ACSII files is done from the "load/save sound" menu. Set Waveform Menu The sound's waveform is determined either synthesized (generated by a function) or is sampled (recorded) from a real sound. The Sonify module is preprogrammed with several simple synthesized waveforms which can be combined to generate realistic musical and sound affects. The waveform is selected in the "waveform" browser. Near the top of the browser are two special waveforms: "silence" and "sampled". The "silence" waveform is as you would expect--no sound at all. The sampled waveform is used to read a presampled sound from a file (in AIFF format) from disk. The file used for the "sampled" waveform is selected with the "sample file" browser. The browser has no affect unless the "sampled" waveform is selected. Synthesized waveforms include: sin step pulse sawtooth triangle noise complex sin Each synthesized sound can accept optional parameters which affect how the waveform is built, giving you an unlimited number of variations that can be generated. The "A", "B", "C", "D", and "E" sliders are used for inputting parameters to the synthesized waveform function. They may produce different results depending on the waveform used (and some may have no affect). Set Envelope Menu The options in the Set Envelope menu are filters that are applied to the sounds before they are summed into the final output sound. The "pitch" increases the frequency of the sound waves by skipping samples. Note that pitch changes will only work until you begin "aliasing" or skipping into samples that actually belong to the following wave cycle. Note also that the pitch can currently only increase the frequency, and cannot (yet) duplicate samples to decrease frequency. The "volume" slider will increase of decrease the overall amplitude of the waveform, effectively controlling it's loudness. The actual volume of each sample is determines by the attack, decay, sustain, and release values. Most sounds do not have a constant volume, and actually change volume as the sound is played based on the following graph: /\ A = attack / \d D = decay / \___________ --S S = sustain level a/ s \ R = release / \r a = attack phase / \ d = decay phase | | | s = sustain phase A D R r = release phase The sound will gradually build to it's maximum volume during the attack phase. The "attack rate" slider determines the position of the end of the attack phase. The maximum volume at the end of the attack phase is determined by the "volume" slider. The "decay rate" slider determines the end of the decay phase in which the sound gradually lowers in volume to the sustain level (determined by the "sustain level" slider). During the sustain phase, the sound remains at a constant volume. The "release rate" slider determines the end of the sustain phase and the beginning of the release phase, in which the volume gradually drops to zero. The portion of the waveform used when summing the final output sound is determined by the "start clip" and "end clip" sliders. The "start place" and "end place" sliders determine where in the summed output sound the current sound should be placed. The sound will always fall within this placement window of the total output time. The clipped sound may be repeated within the placement window with the "repeat on" button, otherwise the waveform will appear only once and may not reach to the end of the placement window. The attack, decay, sustain, and release of the sound are applied in the placement window with the beginning of the attack phase starting at the "start place" and the end of the release phase occuring at the "end place". Load/Save Sound Menu All parameters associated with a particular sound can be saved into a ASCII sound setup file. This file should not be confused with the AIFF files in which sampled sounds are stored. To store a setup file to disk, define the filename with the "sound file" browser. If you are creating a new file, you will have to click on "New File" at the bottom of the browser. Then press the "save file" button. To load a previously stored setup file, find the file in the browser in order to define it's pathname, and click "load file". It may take several seconds for the Sonify module to execute the loaded file since it may have to generate the sound or load in sample files associated with the sound. Some sample sound setup files may be packaged with the Sonify module. They will have a ".snf" extension. Hints Most of the simple waveforms only define one cycle of the sound, so you must have the repeat option on in order for them to be useful. Most of the simple waveforms use parameter "A" to build at a certain pitch, which actually increases or decreases the number of cycles per second rather than simply skipping samples like "pitch". However, unlike "pitch", the individual waveforms must be reconstructed any time a parameter changes. Clipping is more useful on sampled sounds than on the simple waveforms, although with the repeat on, it can give interesting results on simple waveforms as well. INPUTS Command string in (string). Almost everything that can be changed interactively using the widgets can be controlled via commands to the Sonify module's string input. The commands are parsed and executed as if you had moved the corresponding wigit(s) interactively. This allows other modules full control over the capabilities of the Sonify module to produce sound that can correspond to real-time animated data. Possible commands are: begin end mode marker|mapper time channels mono|stereo frequency |none output yes|no loadfile name onoff on|off speaker left|right|both waveform samplefile A B C D E pitch volume attack decay sustain release sclip eclip splace eplace repeat on|off Multiple commands can be strung together separated by a semicolon (;). The soundlist is a list of one or more sounds separated by a comma (,) which are affected by the command. Each specified sound can be either the sound's number, or its name. Keyword "all" may be specified as the soundlist for all sounds. Keyword "current" may be specified as the soundlist for the current sound. Values can be either absolute, or if preceeded by a plus (+) or minus (-) will be considered relative to the current value. The commands themselves may be shortened as long as they are nonambiguous. Here are some examples of sound commands: eclip 1,3,sound-04,7 5.9 changes the end clipping value to 5.9 for sound #1, #3, #7, and the sound named "sound-04". time +0.5 ; repeat 7 on ; vol 9,sound5 50.0 increases the length of the output sound by a half second, turns repeat on for sound #7, and changes the volume of sound #9 and "sound5" to 50. A 5,hello,8 +0.9 increases the "A" parameter of sound #5, "hello", and #8 by .9. The "begin" and "end" commands allow you to make several changes without the current sound changing after each change. Commands following a "begin" command do not affect the current sound until an "end" command is issued. This serves two purposes. One, the user doesn't have to concatenate a lot of commands into a single string in order to make them all take affect at once, and two, due to a shortcoming of AVS, strings are limited in size, so in order to make many changes it is necessary to break the commands up into multiple strings. The "loadfile" command is used to load a sound setup file from disk into the sound(s) in the soundlist. The sound setup files stored with the "save file" option (see "Load/Save Sound Menu" above) are simply ASCII files containing a list of commands separated by commas. It is possible to create a sound setup file using a text editor and execute it with the "load file" button or with the "loadfile" command. OUTPUTS Sound out (field uniform float) The Sonify module can send the output sound to the output port in the form of a 1D or 2D field (depending on whether or not the sound is mono or stereo). The sound's frequency is stored in it's label, and may be extracted with the following code: int freq; AVSfield_float *sound; char label[256]; freq = atoi(AVSfield_get_label(sound, 0, label)); EXAMPLES Sonification network to control a synthesized sound and graph it: Controlling module | | Sonify | | GraphSound | | Geometry Viewer BUGS Due to bugs in the SGI libmalloc routines, the Sonify module sometimes crashes on the SGI for no apparent reason. This seems to happen a lot when using sampled sounds, and when using the "loadfile" command to read in command files. The "pitch" slider should allow the frequency to decrease by duplicating samples, but at the moment, you can only increase the frequency by skipping samples. Also, even though the slider is a float value, the value is currently truncated to an integer so 3.5 and 3.7 would both sound the same. RELATED MODULES GraphSound ReadSound PlaySound GenSound created by Brian Kaplan and John MacCuish kaplan@cica.cica.indiana.edu jmaccuis@cica.cica.indiana.edu Center for Innovative Computer Applications (CICA) Indiana University