The Dyse Builder Sound Server is a sound server that has been built by David Beaudry under Cycling 74's Max/MSP (http://www.cycling74.com).
The vrNav2/Dyse Builder Sound Server combination can be used to add spatialized sounds to models. It can also be used to create highly interactive sound environments in which shound are controlled in the following ways:
All communication between vrNav2 and the Dyse Builder Sound Server are accomplished by a two way exchange of messages that are sent back and forth between vrNav2 and the Sound Server. These messages conform to the DbMaxProtocol.
The Sound Server is started before vrNav2. When vrNav2 starts up on a model that has had sounds added to it, it takes the following actions:
After that, as the model is navigated, at every frame vrNav2 sends the sound server messages:
vrNav2 accepts messages sent to it by the Sound Server about changes it has made. It keeps track of the state of each sound source and group so that other vrNav2 components can query the sound state.
If the Sound Server sends a message asking that a sound be moved, vrNav2 moves the group node to which the sound is attached (thus moving the sound and all the geometry under that group node).
When the user quits vrNav2:
vrNav2 sound processing works with and has been tested only with Multigen Creator .flt model files.
Here are the ways that sound sources can be attached to a model:
Specify /attachtoposition for this type of attachment. Sounds for which /attachtoposition has been specified are all placed together on a branch of the scene graph separated from the rest of the model.
If the attach to node is named:
something_hideor:
hide_somethingWhere "something" is replaced by any string, the attachment node (and anything below it if it is a group node) is removed from the scene graph. This allows the modeler to add nodes for sound attachment to the model which will not be visable when the model is navigated.
To run vrNav2 with the sound server specify the vrNav2 command as follows:
vrNav2 [vrNav2-options] model_config_file.xmlYou must include the following as one of the vrNav2-options:
-sound PATH ADDRESS PORT sound.dbMax
Where:
Replace ADDRESS with the address of the machine that is running your Dyse Builder Sound Server and PORT with the PORT you will use for the communication. The Dyse Builder Sound Server must have already been started on that machine and it must have connected to vrNav2 at its address and using the same port PORT, before vrNav2 can be started.
The sound.dbMax file is the config file for the sounds.
- Replace PATH with the path to the directory containing the sound.dbMax files.
- Replace sound.dbMax with the name of your sound.dbMax file.
The sound.dbMax config file is used to specify the sound arrangement of the model space.
In the sound.dbMax file blank lines are ignored in the file and comments lines are indicated with an initial /*. Lines beginning with / are commands.
The sound.dbMax Config file is organized into parts that must appear in the correct order. Within a part, commands can appear in any order. Some commands are compound commands. Compound commands consist of an initial command and one or more following continuation commands.
In the syntax below:
- <ModelAlias>
- with the name of the model
- <BoundsAlias>
- with the name of a node in the model that defines
a bounds. See the discussion of bounds below.
- <SoundFileAlias>
- with a name you will use to identify a
particular sound file.
- <GroupAlias>
- with a name you will use to identify a sound
source group.
- <SoundSourceAlias>
- With a name you will use to identify a sound source.
Consult the Dyse Builder Sound Server documentation, found in sound_server.pdf
for the documentation on what these commands mean.
For compound comannands,
replace <n> with the number of following continuation commands.
Omit /matersvol and /woofervol unless you want to boost the sounds.
/* Part 1
/modelunits {meters|feet}
/mastervol <float>
/woofervol <float>
/* Part2
/model <ModelAlias> {off|on}
/errorreport {off|on}
The bounds command defines a bounding region used by they Sound Server. For example in the model of a house and its surrounding grounds, the house group node can be used as a bounds. Outside the house there can be a certain sound environment consisting of the sound of birds and of cars going by. Inside the house there can be a sound environment consisting of the sounds of the radio and the sounds of people talking. Inside the house, the kitchen can define an inner bounds. In the sound environment inside the kitchen we hear the water running and the refrigerator cycling on and off.
In order to use bounds, the modeler must arrange the tree structure of the model logically. In the example just given, house and kitchen must be named group nodes. All the all the walls comprising the boundaries of the house must be defined on the house branch of the model. Any objects inside of the house bounds must be on the house branch of the model, i.e., under the house group node. Any objects not in the house must be on other branches of the model's tree and not under the house branch.
Since kitchen is a bounds inside of house, kitchen must be a named group node under the house group node and all objects that are inside of kitchen must be i under it on its branch. Objects that are inside of house but not inside of kitchen must be on the house branch of the model but not on the kitchen sub branch.
The vrNav2 code uses this ordering to determine that a bounds is inside of another bounds and also to determine which bounds the viewer/listener is currently in during model navigation.
In part 3 of the sound.dbMax file code one /bounds command for each bounds your sound model is using. If you are not using bounds, omit part 3.
/* Part3
/bounds <BoundsAlias> <float>
A sound file is an audio file that is stored on the Sound Server. Replace <name> with the name of a sound file such as bell.aif. You can only code the names of sound files that the Sound Server actually has.
A SoundFileAlias is a name that you will use to identify a particular sound file. In part 4 code a /soundfile command for each SoundFileAlias you will use. You man assign multiple SoundFileAliases to the same sound file and you must do so if you associate the same sound file with different sound sources via /load and /def paragraphs in part 6 of the sound.dbMax file.
/* Part4
/soundfile <SoundFileAlias> <name>
...
/soundfile <SoundFileAlias> <name>
Two or more sound sources can be grouped into a sound source group. A sound source group is defined in a group paragraph. Part 5 of the sound.dbMax file consists of zero or more group paragraphs. The format of a group paragraph is a /group command followed by zero or more commands for those sound source attributes that can be specified for a group, in any order. The sound source commands that can be specified for a group are:
/loop <int> <int>
/affectedbybounds {off|on}
/volsourceinit <float>
/end {off|on}
/doppler {off|on}
On the /group command, replace <GroupAlias> with the name you want to give this group.
/* Part5
/group <GroupAlias>
[one or more commands that specify group attributes]
A /load paragraph is used to specify a sound source that vrNav2 is to attach to the model. A sound source is either to be attached to the model at a specific x,y,z or by being attached to a named node in the model.
A /def paragraph assigns sound attributes to sound nodes that have been placed in the model by the modeler by creating group nodes named _Sound_NAME where NAME is any name. There can be /def paragraphs in the sound.dbMax file for which there are no sound nodes in the model. These will be ignored.
There are different kinds of /load paragraphs:
- /load
- A sound source associated with a single sound file.
- /load_mix
- A sound source associated with multiple sound files that are to be mixed.
- /load_chain
- A sound source associated with multiple sound files that are to be played one after the other.
- /load_live
- A sound source to be associated with a live microphone.
Similarly, a /def paragraph can either be /def, /def_mix, /def_chain, or /def_live.
Part 6 of the sound.dbMax file consists of all the /load and /def paragraphs in all their variations.
The /load and /def paragraphs each consists of:
The syntax of the /load and /def commands is:
/load <SoundSourceAlias> <SoundFileAlias>
/def <SoundSourceAlias> <SoundFileAlias>
Replace <SoundSourceAlias> with a name that your will use to identify this sound source. Replace <SoundFileAlias> with a sound file alias you specified if Part 4. The sound file that you name will be the one that will play at this sound source in the model.
The following commands can be used to specify sound file attributes:
/playpoints <float> <float>
/playspeed <float>
/startfade <float> <float>
/stopfade <float> <float>
/fadeEOF {off|on}
/out <n>
/volinit <float>
Consult the Dyse Builder Sound Server document (.pdf file) for the meaning of these attributes. /out is a compound command whose syntax is:
/out <n>
/mapping <int> <float> <float> <float>
...
/mapping <int> <float> <float> <float>
The following commands can be used specify sound source attributes:
/loop <int> <int>/autostart {off|on}
/playwhenin <BoundsAlias> <int> <int> /playwhenwithin <float> <float> <float> <float> <float> <float> <int>/affectedbybounds {off|on}
/point <float>
/background/attachtogroup <GroupAlias>
/volsourceinit <float>
/end {off|on}
/startfadesource <float> <float>
/stopfadesource <float> <float>
/doppler {off|on}
/movable {off|on}/attachtoobject <string>
/attachtoposition <float> <float> <float>
Consult the Dyse Builder Sound Server document (.pdf file) for the meaning of these attributes. /autostart, /playwhenin, and /playwhenwithin are mutually exclusive. Only the first one to appear in the paragraph will take effect. Similarly, /point and /background are mutually exclusive. Only the first one to appear in the paragraph will take effect.
Either /attachtoobject or /attachtoposition must be specified.
Here is a sample /load paragraph
/* Part6
/load bell bell
/* sound file attributes
/playpoints 4.0 0.0
/playspeed 2.5
/volinit 6.0
/* sound source attributes
/autostart off
/loop 5 50
/point 1.0
/end on
/attachtoposition 0.0 50.0 20.0
/startfadesource 10.0 0.0
/attachtogroup train
The syntax of the /load_live and /def_live paragraphs is similar to that of the /load and /def paragraphs except that, there is no sound file associated with the sound source and the command specifies the number of the input channel instead. Also, since there is no sound file, there are no sound file attributes. Here is an example:
/load_live naration 1
/* sound source attributes
/autostart off
/end on
/attachtoposition 0.0 0.0 0.0
The syntax of the /load_mix and /def_mix paragraphs is similar to that of the /load and /def paragraphs except that, since there are multiple sound files, the /load_mix and /def_mix commands include the SoundSourceAlias and the number of following /mix commands and the SoundFileAliases are each specified on a separate /mix command. Here is an example:
/load_mix noises 3
/* now we must supply 3 /mix commands
/mix talking 0
/mix boots 5
/mix crickets 8
/* sound file attributes
/playspeed 2.5
/* sound source attributes
/playwhenwithin 1.0 1.2 .13 2.3 2.5 6.0 1
/background
/end on
/attachtoobject BaseGeo
The syntax of the /load_chain and /def_chain paragraphs is similar to that of the /load_mix and /def_mix paragraphs except that the SoundFileAliases are specified with /chain commands and each one can have a set of sound file attributes. Here is an example:
/load_chain lould_serial_noises 3
/* now we must supply 3 /chain commands
/chain hammer 14.0
/*this sound file may have attributes associated with it
/volinit 6.0
/chain truck 25.0
/*this sound file may have attributes associated with it
/volinit 4.0
/chain argument 0.0
/*this sound file may have attributes associated with it
/playpoints 4.0 0.0
/playspeed 2.5
/* sound source attributes
/autostart off
/end on
/attachtoposition 400.0 80.0 0.0
/attachtogroup outside
vrNav2 currently has a number of keys reserved for use with the sound server. We used these keys to test what would happen if the user requested that a sound be muted, or have its volume raised, or that all sounds be muted, etc. We intend to add a menu to vrNav2 that would allow the user to select as sound or sounds and make changes to it. Our DbMaxProtocol currently has the following functions that the vrNav Program or other program can call to set or query sounds:
Queries that return the names of Sound Sources and Groups:
// returns a vector of all the SoundSourceAliases in the model
std::vector< std::string >&
getSoundAliases( );
// returns a vector of GroupAliases in the model
std::vector< std::string >&
getGroupAliases( );
Queries for specific Sound Sources and Groups. In each case alias is either a Sound Source alias or a Group alias:
// Returns the play status for a Sound Source or Group
ESoundStatus
getStatus( const std::string& alias ) const;
EOffOn
getLoop( const std::string& alias ) const;
// Returns the time between loops for a Sound Source or Group
int
getTimeBetween( const std::string& alias ) const;
// Returns the number of times the loops repeats
int
getLoopNRepeats( const std::string& alias ) const;
// Returns the end notification associated with a Sound Source or Group
EOffOn
getEndNotificationRequested( const std::string& alias ) const;
// Returns the doppler associated with a Sound Source or Group
EOffOn
getDoppler( const std::string& alias ) const;
// Returns the Bounds alias of the Bounds currently associated with a
// Sound Source or Group. For a Group to be within a Bounds, all members
// of the Group must be within that Bounds.
std::string
getBoundsAlias( const std::string& alias ) const;
float
getVolLevel( const std::string& alias ) const;
Queries for specific Sound Sources:
// Returns whether the Sound Source is a point or background Sound Source
EPointOrBack
getPointOrBack( const std::string& alias ) const;
// Returns the scale associated with a point Sound Source
float
getPointScale( const std::string& alias ) const;
float
getFadeInTime( const std::string& alias ) const;
float
getFadeInShape( const std::string& alias ) const;
float
getFadeOutTime( const std::string& alias ) const;
float
getFadeOutShape( const std::string& alias ) const;
// Returns whether a Sound Source is attached to a movable object
EOffOn
getMovable( const std::string& alias ) const;
Queries for Sound File attributes of Sound Files associated with Sound Sources. In each case fileAlias is the SoundFileAlias and alias is the SoundSourceAlias:
bool
getPosition( const std::string& alias,
float& x, float& y, float& z ) const;
bool
getOrientation( const std::string& alias,
float& h, float& p, float& r ) const;
float
getStartPlay( const std::string& alias,
const std::string& fileAlias ) const;
float
getEndPlay( const std::string& alias,
const std::string& fileAlias ) const;
float
getPlaySpeed( const std::string& alias,
const std::string& fileAlias ) const;
float
getFadeInTime( const std::string& alias,
const std::string& fileAlias ) const;
float
getFadeInShape( const std::string& alias,
const std::string& fileAlias ) const;
float
getFadeOutTime( const std::string& alias,
const std::string& fileAlias ) const;
float
getFadeOutShape( const std::string& alias,
const std::string& fileAlias ) const;
EOffOn
getFadeEOF( const std::string& alias,
const std::string& fileAlias ) const;
float
getVolLevel( const std::string& alias,
const std::string& fileAlias ) const;
Functions that make global changes:
bool
changeAudprocToOff( );
bool
changeAudprocToOn( );
Functions that can be called to change the position or orientation of movable sounds:
bool
setMovablePosition( const std::string& alias,
const float& x,
const float& y,
const float& z );
bool
setMovableOrientation( const std::string& alias,
const float& h,
const float& p,
const float& r );
Functions that change the attributes of Sound Sources and Groups:
bool
changeStatusToStopPause( const std::string& alias,
const ESoundAction& action );
bool
changeStatusToPlayResume( const std::string& alias,
const ESoundAction& action );
bool
changeLoopToOff( const std::string& alias );
bool
changeLoopToOn( const std::string& alias,
const int& between, const int& nRepeats );
bool
changeVolLevelTo( const std::string& alias, const float& level );
bool
changeEndNotificationTo( const std::string& alias,
const EOffOn& val );
bool
changeDopplerToOff( const std::string& alias );
bool
changeDopplerToOn( const std::string& alias );
Functions that change the attributes of Sound Sources:
bool
changeFadeInTo( const std::string& alias,
const float& time, const float& shape );
bool
changeFadeOutTo( const std::string& alias,
const float& time, const float& shape );
Functions that change the Sound File attributes of Sound Files associated with specific Sound Sources. In each case fileAlias is the SoundFileAlias and alias is the SoundSourceAlias:
bool
changeStatusToStopPause( const std::string& alias,
const std::string& fileAlias,
const ESoundAction& action );
bool
changeStatusToPlayResume( const std::string& alias,
const std::string& fileAlias,
const ESoundAction& action );
bool
changePlayPointsTo( const std::string& alias,
const std::string& fileAlias,
const float& start, const float& end );
bool
changePlaySpeedTo( const std::string& alias,
const std::string& fileAlias,
const float& speed );
bool
changeStartFadeTo( const std::string& alias,
const std::string& fileAlias,
const float& time, const float& shape );
bool
changeStopFadeTo( const std::string& alias,
const std::string& fileAlias,
const float& time, const float& shape );
bool
changeFadeEOFTo( const std::string& alias,
const std::string& fileAlias,
const EOffOn& offon );
bool
changeVolLevelTo( const std::string& alias,
const std::string& fileAlias,
const float& level );
vrNav2 communicates with the Dyse Builder Sound Server via UDP. All message packets conform to the OpenSound Control Specification that can be found at http://cnmat.cnmat.berkeley.edu/OSC/OSC-spec.html with the exception that compound bundles are not supported, i.e., messages are sent individually or in bundles but bundles containing other bundles are not supported on the vrNav2 side.
On startup, vrNav2 sends the following message to the Dyse Builder Sound Server:
Then it sends the information contained in parts 1 and 2 of the sound.dbMax file. The messages sent are as follows:
/modelunits {0|1}
/mastervol <float>
/woofervol <float>
/model <ModelAlias> {0|1}
/errorreport {0|1}
Then it sends the information about all the sound files contained in part 4. The messages sent are:
/soundfile <SoundFileAlias> <string>
Then it sends the information about all the Groups contained in part 5. The messages sent are:
/group <GroupAlias>
Each /group message is followed by zero or more of the following messages:
/loop_g <GroupAlias> {0|1} <int> <int>
/affectedbybounds_g <GroupAlias> {0|1}
/volsourceinit_g <GroupAlias> <float>
/end_g <GroupAlias> {0|1}
/doppler_g <GroupAlias> {0|1}
Then vrNav2 sends all the information about all the Sound Sources specified by /loads to the Dyse Builder Sound Server. Each set of messages begins with:
/soundsource <SoundSourceAlias>
This is followed by zero or more of the following messages:
/autostart <SoundSourceAlias> {0|1}
/playwhenin <SoundSourceAlias> <BoundsAlias> <int> <int>
/playwhenwithin <SoundSourceAlias> <float> <float> <float> <float> <float> <float> <int>
/loop <SoundSourceAlias> {0|1} <int> <int>
/point <SoundSourceAlias> <float>
/background <SoundSourceAlias>
/volsourceinit <SoundSourceAlias> <float>
/end <SoundSourceAlias> {0|1}
/attachtogroup <SoundSourceAlias> <GroupAlias>
/startfadesource <SoundSourceAlias> <float> <float>
/stopfadesource <SoundSourceAlias> <float> <float>
/doppler <SoundSourceAlias> {0|1}
/movable <SoundSourceAlias> {0|1}
/affectedbybounds <SoundSourceAlias> {0|1}
/playpoints <SoundSourceAlias> <SoundFileAlias> <float> <float>
/playspeed <SoundSourceAlias> <SoundFileAlias> <float>
/startfade <SoundSourceAlias> <SoundFileAlias> <float> <float>
/stopfade <SoundSourceAlias> <SoundFileAlias> <float> <float>
/fadeEOF <SoundSourceAlias> <SoundFileAlias> {0|1}
/out <SoundSourceAlias> <SoundFileAlias> <n> [<int><float><float><float>] <-nTimes
/volinit <SoundSourceAlias> <SoundFileAlias> <float>
And then one of:
/load <SoundSourceAlias> <SoundFileAlias>
/load_live <SoundSourceAlias> <int>
/load_chain <SoundSourceAlias> <n> [<SoundFileAlias> <float>] <-nTimes
/load_mix <SoundSourceAlias> <n> [<SoundFileAlias> <int>] <-nTimes
After sending the information about all the sounds that were specified with /loads, vrNav2 sends similar information about all the sound sources that were found on the model. The information is taken from the associated /defs found in the sound.dbMax file.
vrNav2 next sends the /modelbounds messages as follows:
/modelbounds <float> <float> <float> <float> <float> <float>
Where the 6 floats are the x,y,z extents of the bounding box containg the model.
Next vrNav2 sends:
/attachtobounds <GroupAlias> <BoundsAlias>
/attachtobounds <SoundSourceAlias> <BoundsAlias>
messages for each Group and Sound Source that are attached to Bounds. The <BoundsAlias> specified is the alias of the Bounds that vrNav2 finds the Sound Source or Group in on the scene graph for the model. A Group is inside a Bounds only if all of the Sound Sources in the Group are contained within that Bounds.
After sending one set of the messages described in the next section, the final message that vrNav2 sends to the Dyse Builder Sound Server at start up is:
/audproc 1
Messages sent to Dyse builder every frame are:
/viewerpos <float> <float> <float> <float> <float> <float>
Where the floats are the x,y,z h,p,r of the viewer/listener in the model space.
/viewerbounds <BoundsAlias>
Where <BoundsAlias> is the alias of the Bounds the viewer/listener is in or Model if the viewer/listener is outside of all the Bounds.
Then for each Sound Source that has moved the following message is sent:
/sourcepos_cart <SoundSourceAlias> <float> <float> <float>
Where the three floats are the x,y,z position of the Sound Source in model space. If the Sound Source is movable:
/move_dcs <SoundSourceAlias> <float> <float> <float> <float> <float> <float>
These floats are x, y, z, h, p, r. They indicate the location of the object in the model to which the sound source is attached. If the Dyse Builder Sound Server wants to move the sound, it needs to know the current position in model space of the sound and of its attachment point in order to compute how to change the position of the attachment point to move the sound. All geometry attached to and below the attachment point, will be moved along with the sound.
vrNav2 receives and acts on any messages that the Sound Server has sent it at each frame. These can include: /audproc, /loop, /loop_g, /volsource, /volsource_g, /end, /end_g, /playbacksource, /playbacksource_g, /doppler, /doppler_g, /startfadesource, /stopfadesource, /setsourceposition, /playpoints, /playspeed, /startfade, /stopfade, /fadeEOF, and /vol.
The syntax of most of these messages has been given previously. The ones that haven't are:
/playbacksource <SoundSourceAlias> <int>
/playbacksource_g <GroupAlias> <int>
/setsourceposition <SoundSourceAlias> <float> <float> <float> <float> <float> <float>
/volsource <SoundFileAlias> <float>
/vol <SoundSourceAlias> <SoundFileAlias> <float> <float> <float>
The Dyse Builder Sound Server sends most of these messages to vrNav2 to inform it that the Sound Server has made some change. Then when one of the get functions listed in section 7 is called, the up to date information can be returned to the caller. It is imperative that the Sound Server keep vrNav2 informed in this way.
The exception is /setsourceposition. The Sound Server sends this message to instruct vrNav2 to move a Sound Source on the scene graph for the model. vrNav2 moves the sound source by moving the node it is attached to. If that node is a group node, all parts of the model that are handing from that group node will move as well.
vrNav2 sends the following messages to the Sound Server when the user quits the vrNav2 application:
/audproc 0
/reset
This instructs the Sound Server to turn off all audio processing and to reset itself for another model.