HLP-R Dialogue Production **************************** The dialogue module of hlpr_dialogue_production can be used to integrate a dialogue state machine into your code. You can also run the standalone server in standalone_server.py and use the DialogueAct action (defined in this package) to send dialogue requests to the robot. Writing Dialogue and Adding Behaviors ====================================== The dialogue production code is based on `CoRDial TTS `_, from the USC Interaction Lab, which uses Amazon Polly for speech synthesis. You will need an AWS account in order to use Polly. This code parses strings with simple `<>` tags for behaviors that should be synchronized to the speech. These tags take the form `` and will be synchronized to the beginning of the first word that folows the tag. Although a mechanism for adjusting the timing on a per-behavior basis is provided, this class is not recommended for cases where tight (within 300ms) synchronization is required, since lag in the ROS message system can affect the precise timing. In practice, this works well for most HRI contexts where behavior timing itself isn't being studied. CoRDial TTS also automatically extracts visemes and passes them as behaviors to the dialogue system, although these are not currently handled for HLP-R. Each behavior type takes a controller that provides an interface to an action server that executes the behavior. How the behaviors should be allocated among controllers depends on the interaction context and what clashes you expect to have between degrees of freedom, but the currently-provided controllers are as follows: * Lookat Controller: Moves the robot's head to look at a point * Behaviors: * lookat * Arguments: * If 1 argument: the frame to look at * If 4 arguments: a frame and position relative to that frame to look at * Gesture Controller: Moves the robot's arm through predefined key frames * Behaviors: * wave * shrug * thinking * Arguments: * *none* * Test Controller: Plays a beep sound * Behaviors: * test * Arguments: * *none* You can also provide a "phrase file" to the system, which is a text file where each line is a dialogue act with behaviors in angle brackets, beginning with a unique key in square brackets:: [ex1] This is a phrase file entry! The phrase file can be used to pre-load audio files using ``gen_phrases.py``. Run ``gen_phrases.py --help`` for usage information. Running the Stand-Alone Server -------------------------------- Before you run the server, you will need to have an AWS account and enable your computer to access Polly using Boto3. Some basic instructions can be found `here `_. 1. Check out the "catkin" branch of `CoRDial `_ and place the cordial_tts package in the src folder of your catkin workspace 2. Make the messages and services by running ``catkin_make`` in your workspace root directory 3. Launch the HLP-R/Poli simulation, with the Lookat waypoints server and MoveIt! enabled 4. Launch the controllers and action server:: roslaunch hlpr_dialogue_production all_controllers.launch 5. Test the setup:: rosrun hlpr_dialogue_production test_action_client.py You should hear speech through your computer speakers and see the robot move. For more information on the options to the stand-alone action server, run ``rosrun hlpr_dialogue_production standalone_server.py --help``. Adding Controllers ------------------- The currently-implemented controllers can be found in ``controllers.py``. A ControllerState provides an interface to an ActionServer, providing a callback function that turns the string behavior name and string list of arguments into a goal for that server, and calling the server at the appropriate times relative to the speech audio. If you would like to implement a controller that you think many folks could use, add it to ``controllers.py`` by adding the following functions, substituting the name of your controller for ``[your]``:: [your]_controller_cb(behavior_name, string_args) get_[your]_controller() You can then add it to the action server by adding it to the list starting on line 71 of ``standalone_server.py``. .. warning:: Make sure that neither the state name (given in all caps by smach convention) nor the callback names clash with any of the existing controllers!! Code API =================================== Controllers ----------- .. automodule:: hlpr_dialogue_production.controllers :members: Main Smach Wrapper Class ---------------------------------- .. currentmodule:: hlpr_dialogue_production.dialogue .. autoclass:: SmachWrapper :members: __init__, get_active_states, get_outcome, get_sm, is_running, preempt, reset, standalone_start Smach States and Related Classes --------------------------------- .. automodule:: dialogue .. autoclass:: Synchronizer :members: __init__, start, reset .. autoclass:: ControllerState :members: __init__, setup_sync .. autoclass:: TTSSpeechStart :members: __init__ .. autoclass:: TTSFallbackSpeechStart :members: __init__ .. autoclass:: FileSpeechStart :members: __init__ .. autoclass:: NoPrepSpeechStart :members: __init__ .. autoclass:: SpeechState :members: __init__