18 March 2011 

                        dSurvey's parameter files.

                        ===================
1)          Introduction
2)          Basic structure
3)          Syntax
3a) Compressed syntax
3b) Expanded syntax for REFERENCE
Appendix 1) Another way of considering the kinds of Parameter Files
Appendix 2) Example of a parameters file
Appendix 3) Internal storage of data

                        ===================
1) Introduction

  dSurvey uses XML to store parameters for each session.

  dSurvey has two classes of parameter files:
   * administrative. This contains parameters read, and used, by control panel programs.
        Administrative parameter files specify details of the survey insrument, and of
        participants enrolled in the survey. 
        
   * client-accessible. These contain parameters that are provided to clients.
         Note: by "client", we mean "the javascript program running on a particpant's machine".
         Thus, client is more or less synonymous with "participant", but suggests that the
         actual human participant does not directly interact with the information.
        
This document focuses on admiminstrative parameter files -- on files that can and should
be changed to reflect changes in the survey instrument, and changes in values 
participants will be working with.

  Note that client-accessible parameter files have  very similar structure. 
  But since these are automatically generated, there is little reason for 
  an administrator to work with them. 

2) Basic structure
  A dSurvey parameter file can contain several types of variables:
    1) "General". These are variables that are the same for all clients
    2) "User". These are variables that differ across clients
    3) "Reference". These are "reference" data, that is identified by an reference-id. This identifier is NOT necessairly
                     specific to a client. However, a client may be assigned a client-id that can be used to lookup a vector
                     of variables (from the reference data) with a matching reference-id

  Typically:
        * administrative parameter files typically do NOT contain "user" information.
   
  Administrative parameter files are stored in a session specific permanent file directory.

      For example, if the session's name is "sessionName", they will be in /dsurvey/sessions/sessionName

      Instrument specfic "defaults" of the administrative parameter files are specified in the  "instrument descriptor" 
      file. And when a session is created, "session specfic" versions of these these "instrument specific" parameter files are 
      created. Thus: the session-specific administative parameter files are basically copies (with small changes)
      of the instrument specific administative parameter files.

      An instrument (hence sessions based on this instrument) can have more than one administrative parameter file. 
      For example, one file may contain "general" parameters, and another may contain "reference" data. Note that this is a convenience: 
      it is quite legal to  have one file contain both kinds of data.

      Note that administrative parameter files are NOT read by clients. Hence, this "sessionName" directory does NOT
      need to be web accessible.

   Technical note:
      Client-accessible parameter files are created by the session's survey control program.
         Often, they are close copies of the session specific administrative parameter files.
         Or, they may contain client-specific values of reference data.
         They are also used to transmit intermediate results back to the participants.
      The client-accessible parameter files are stored in the dsurvey/pinfo/sessionName directory.
        This directory MUST be web accessible!

3) Syntax

Administrative parameter files are legal XML files. 

   * However, some  "compression" shortcuts are used This saves space, but
    limits what information can be stored. 

If this is a problem, an expanded syntax is available

  * The expanded variant uses a more standardized XML format. Its more verbose, but can
    store large chunks of information.

In both cases, text information is stored. Binary data, such as images, are NOT supported.

                =============

3a) Basic syntax

The overall structure is:

 <?xml version="1.0" encoding="UTF-8" ?>
  <parameters>
      <general>
          <desc name="var1"> a description </desc>               :: desc is optional. var1 etc will be matched with a <var1>.
          <var1> .. </var1>
                 ...
           <desc name="var2"> a description </desc>               :: each var can be "described" by its own <desc>
           <var2> .. </var2>

           <varj array="1" delim="x" > .. , .. , .. </var1>                   :: specify a "x" ("," if not specified) delimited array of values

            <vark element="n">aval </vark>
            <varN> .. </varN>
      </general>

      <users>                                            ::  <USERS>  are NOT typically specified in an administrative parameters file 
          <vars > vname1,...,vnameM </vars>                :: comma, or space, delimited list of variable names
          <descriptions>                                   :: optional.
             <var name="vname1"> a description  </var>              :: vname1 is matched with a variable defined in <vars>
                        ...
             <var name="vnameJ">   a description  </var>            ::  it is NOT required to have a "description" for all the                                                                                vnames specified in <vars>
          </descriptions>
          <vals delim=",">                                           :: , used as delimiter
               <user id="clientID1" >val1,val2,...,valM </user>    :: the clientId is unique to each client
                                   ...
               <user  id="clientIDN">val1,val2,...,valM  </user>    
          </vals>
      </users>

       <reference>
          <vars > vname1,...,vnameM </vars>                :: comma, or space, delimited list of variable names
          <descriptions>                                   :: optional.
             <var name="vname1"> a description  </var>              :: vname1 is matched with a variable defined in <vars>
                        ...
             <var name="vnameJ">   a description  </var>            ::  it is NOT required to have a "description" for all the vnames specified in <vars>
          </descriptions>
          <vals >                                          :: delim is not specified, so a ' ' is used
             <record id="refId" > val1 val2 ... valM </record>        :: refId is often matched against a REFID assigned to a client
                             ...
             <record id="refId"> val1 val2 ... valM  </record>
          </vals>
       </reference>

 </parameters>

 In addition, for descriptive purposes, a <modified> or <created> element may be inserted at the top of the file.

 Notes:
    *  The array and delim attributes in the <general> section are used to specify arrays of values.
       Delim is optional: it specifies what seperates the values (a comma is used by default)
       array="1" means that an array will be created, whose first value will be at [1]
       array="0" means that an array will be created, whose first value will be at [0]
                
       Example:
            <discounts array="1">0.65,0.8,0.8,0.8</discounts>
        An array with a name of "discounts" will be created, with the first value stored in discounts[1]. 
        Thus, discounts[0]=undefined, discounts[1]=0.65, discounts[2]=0.8, discounts[3]=0.8, and discounts[4]=0.8

   *  The element attribute in the <general> section is used to specify an element of an array.
      This is  hack that deals with XML's dislike for tagnames of the form <Z[1] .. >
      Thus, to specify the 4th element of the V3 array, use:
                <V3 element="4"> ... </V3>
      Note that this syntax is something of an alternative to using the array attribute.

   * The <desc> elements, in <general>, are optional. They are used by "parameter file" editing programs for informative purposes.
     In fact, to save space, client-accessible parameter files typically do NOT contain <desc> elements.

  *  <descriptions> in <user>, and in <reference>, is optional. Their use is similar to <desc> (it is used by editing programs).

  * The syntax of <users> and <reference> is very similar.
    There are a few differences:

          * For not real good reasons, <user> is used inside of <users>, while <record> is used inside of <reference>. Sorry.

          * Each value in a <user> is seperated by a comma

          * By default, each value in a <record> is seperated by a space. However, you can use the delim attribute to change
            the seperator. For example, delim="," means "use a comma as a seperator".

           * clientId is an id that is unique for each client. The client's will find their line by matching their clientID to the
            <user> with a matching ID.
              Dishonest and technically astute participants clients COULD read other lines, and thereby know information on other
              participants. This is a security risk that's endemic to dSurvey.

          * refid is an id that is matched to a REFID variable assigned to each client (for example, as controlled by the HOW_ASSIGN variable
            in the session descriptor file). See  generateParameters.htm for the details.

                =============

3b) Expanded syntax for REFERENCE

You can specify <reference> using an expanded syntax.

  <reference expanded="1" >
          <desc name="var1"> a description </desc>               :: desc is optional. var1 etc will be matched with a <var1> in a <reference>.
                ..
          <desc name="varn"> a description </desc>               

          <record id="refID1" >                                :: the id must be unique to each record
                <var1> ... </var1>
                        ...
                <varn> ...  </varn>
          </record>
                        ...
           <record id="refID2" >                               
                <var1> ... </var1>
                        ...
                <varn> ...  </varn>
          </record>
        
      </reference>

 Specifying array variables.
       You can specify array variables using two methods:
           a) Use a "varn" of the form varname_ivalue_.
              For example, <COST_3_> ... </COST_3_>  will be saved as COST[3]
           b)  Use an index attribute in the element.
                For example <COST index="2"> ... </COST>  will be saved as COST[2]

 Notes:
   * You must include an expanded="1" in the <REFERENCE>.
   * The <desc> elements is similar to its usage in <general>. 
   * Variables inside of  <reference> is similar to inside of <general>, with the tagname of an
     element used as the variable name, and the element's contents used as the value.
     
   * You can specify different variables in different  <RECORD>s.
     However, dSurvey will save the same variables for all   <RECORD>s --
     a '' value is used when a <VARIABLE> was not included in a <RECORD>.


        =======        =======       =======          =======               =======

Appendix 1) Another way of considering the kinds of Parameter Files

   One can identify three "levels" of parameter files.

        a) instrument specific "adminstrative" parameter files.
           These are "default" parameter files specific to this instrument.
           They are never actually seen by administrators or participants. Instead, they are used
           to create session specific administrative parameter files...

        b) session specific "adminstrative" parameter files.
           These are constructed (when a session is created) by copying (with slight modifications) the instrument specific
           adminstrative parameter files.
           The administartor can modify these before the session starts (i.e.; before participants join the session).
           Once a session starts, they are rarely modified.

        c) Transient, client-accessible parameter files
           These are what clients see -- clients do NOT have access to instrument or session specific parameter files
           These represent on-the-fly changes to the parameter files, as dicated by the administrator using the session's control panel.

           It is often useful to use two kinds of client-accessible parameter files: session specific and round specific.

             *   "sesssion specific, client accessible" parameter files are similar to "session specfic, administative" parameter files, but they
                  are often kept up to date (changes to parameters are recorded in the "client accessible session specific" parameter files,
                  but not in the "session specfic  admniistrative" parameter files

             *  "round specific, client accessible" parameter files often contain just the parameters that have recently changed.
                 These are designed to be small, so as to speed up repeated transmission to numerous clients.

=======
Appendix 2) Example of a parameters file:

<?xml version="1.0" encoding="UTF-8" ?>
   <parameters>
   <general>
          <desc value="NACCEPT"> # of tickets to accept. If lt 1.0, fraction of offered tickets to accept</desc>
      <NACCEPT>6</NACCEPT>
          <desc value="TYPENAMES"> Comma delimited list of group "names"</desc>
      <TYPENAMES  array="1">A,B</TYPENAMES>
          <desc value="COST_MIN"> Minimum of the ticket costs.  Comma delimited list, corresponding to group "types".</desc>
       <COST_MIN array="1">0,50,100,150</COST_MIN>
          <desc value="COST_MAX">  Maximum of the ticket costs. Ticket costs will be randomly chosen between COST_MIN to COST_MAX.  Comma delimited list, corresponding to group "types".</desc>
      <COST_MAX  array="1">100,150,200,250</COST_MAX>
          <desc value="BID_MIN"> Lower bound of multiplier to generate "maxbid for this ticket" -- a random multiplied is drawn betweeen bid_min and bid_max, and applied to the ticket's cost</desc>
       <BID_MIN  array="1">1,1,1,1</BID_MIN>
   </general>
   <reference>
         <vars> Group1 Fee1 Group2 Fee2  </vars>
         <descriptions>
            <var name="group1">Group of ticket 1 </var>
            <var name="group2">Group of ticket 2 </var>
            <var name="fee1"> Submission fee for ticket 1  </var>
            <var name="fee2"> Submission fee for ticket 1 </var>
         </descriptions>
          <vals>
             <record id="0">2 2 1 1 </record>
             <record id="1">1 1 2 2 </record>
             <record id="2">2 2 1 1 </record>
             <record id="3">1 1 2 2 </record>
          </vals>
   </reference>
 </parameters>


Or, using expanded syntax

   <reference expanded="1" >
      <desc name="group1">Group of ticket 1 </desc>
      <desc name="group2">Group of ticket 2 </desc>
      <desc name="fee1"> Submission fee for ticket 1 </desc>
      <desc name="fee1"> Submission fee for ticket 1 </desc>

      <record id="0">
          <group1> 2 </group1>
          <fee1> 2 </fee1>
          <group2>1  </group2>
          <fee2> 1 </fee2>
       </record>

       <record id="1">
          <group1> 2 </group1>
          <fee1> 2 </fee1>
          <group2>1  </group2>
          <fee2> 1 </fee2>
       </record>

       <record id="2">
          <group1>1 </group1>
          <fee1> 2 </fee1>
          <group2>2  </group2>
          <fee2> 1 </fee2>
       </record>

       <record id="3">
          <group1>1  </group1>
          <group2>3  </group2>
          <fee1> 12 </fee1>
          <fee2> 5 </fee2>
       </record>
   </reference>




=======
Appendix 3) Internal storage of data

dSurvey will store general, reference, and user data in global objects. Ambitious programmers
can directly access these objects (say, in custom functions used within your control panel).
The following describes the syntax of these variables.

General Variables:

    * general variables are saved in the generalVars control-panel database. 
      Hence, a "costFactor" variable would be saved as generalVars['COSTFACTOR']
   
    *  generalVars['$VARS'] contains a list of all the general variables.

      Actually, a 3 element array is saved to generalVars['VARNAME']:
         * generalVars['VARNAME'][0] : a value of 1 if this should be treated as an array
         * generalVars['VARNAME'][1] : value. If an array, values are always seperated by a comma. 
           Note that the value is always a string (not an array)
         * generalVars['VARNAME'][2] : uncapitalized version of the varname
      You can use readGeneralVar() and getVarInfo() to work with generalVars.

      Note: the getVarInfo function is an easy way of extracting the value of a general (or a control) variable.

User variables:

    * user variables are saved in the userData control-panel database. 
      Hence, for userId of 35, a "income" variable would be saved as 
          userData['35']['INCOME']
      (note the uppercase is always used in object references).

    *  userData['$VARLOOKUP'] is an object used to identify if an variable exists: i.e.; 
          userData['$VARLOOKUP']['COST']=1 if the COST variable exists.

    * userData['$IDS'] contains a list of all the ids.
    * userData['$VARS'] is an array, with each element one of the variables

    * Note: uppercase is always used when specifying variables in userData

Reference variables: 

    * reference variables are saved in the refData control-panel database. 
      Hence, for refId of 12, a "GROUP1" variable would be saved as refData['12']['GROUP1']

    * refData['$VARLOOKUP'] is an objecct used to identify if an variable exists: i.e.; 
           refData['$VARLOOKUP']['COST']=1 if "COST" variable exists 

    * refData['$IDS'] contains a list of all the ids.

    * refData['$VARS'] is an array, with each element one of the variable names


    * Note: uppercase is always used when specifying variables in refData

=======
Appendix 3)  Special attributes in GENERAL data

To assist in displaying/changing GENERAL data, and to help in the creation of scenario files,
you can add the following attributes to any parameternode inside of a <GENERAL> ... </GENERAL>.

The attributes are:
   allowed="val1,val2,..."
        Val,.. should be comma delimited list of allowed users. Some utilities (such as the mkScenario
        scenario creation utility) will read this attrbuteand ONLY permit the parameter value to be equal
        to one of the values (val1, or val2, or ...)
   allowedDesc="desc1, desc2, .."
        Used with allowed="..".  Instead of displaying the values in allowed, display the values in
        allowedDesc. Thus, the number of values specified (using , to seperate values) should be the same
        in allowed and allowedDesc. Note that if allowed is specified but allowedDesc is not, then
        allowedDesc is set equal to allowed
   indent="nnpx"  
         Use by mkScenario.htm. Indent the name displaye by nn pixels. This can be used to make an outline type
         of display of paramters. That is, to indent parameters that can be thought of being in a subset
         of related parameters.
   runtime="1" : if this appears in a scenario file: when the fscenario file is loaded, dSurvey will display
           this (and any other runtime="1" variable  in  a box, and allow you change the value before doing
           anything else.


For example, in the above example above, you can add an attribute

    <NACCEPT>6</NACCEPT>
  
**

