Notes on IVRS/Speech Recognition Project Management

IVRS project management

Caller centric design

Requirement and Design

Requirements --> Design --> Implementation --> Testing --> Deployment --> Optimization

Requirements (designers help to define)

Caller Requirements 

- who, why, ehn, what they known, what need to be told, what they want to do
Business Requirements - do we need to identify caller, what information to be pulled from profile, agent availability,
System Requirements - caller phone no. availability, how long get info from DB, how long the call onhold

Key dimensions of success (specif goal)

Effectiveness - (callers finish what they start) task completion rate, call handle rate
Efficiency - (callers finish without wasting time) average turns, time per transaction
Attractiveness - (caller enjoy?) caller survey, opt-out rates
Utilities (caller need / want to service offered) - call volume (for specifc caller group or task)

Requirement --> drive design decision

Requirement Spec

(simple to avoid scope creep)
- business requirements
- caller requirements
- application requirements
- success metrics
- project deliverables /w deadlines
- team and contact information

Detail of design

- mapping call flow
- mork init and retry prompt
- vocabulary and grammar (for speech recognition)
- confirmation, disambiguation, and logging strategy

Outstanding caller experience:

- Accuracy (best tech, capture caller input and properly recognise and utilize
- low cognitive lead (simple dialog design, intutive, match caller mental model, bridge gap)
- Efficiency and clarify (caller know the progress, clear deliver of information (concise))
- Naturalness
- Graceful Error Recovery - acknowledge, prompt to correct, allow connect to operator
- Persona - define caller relationship with the system/company. match the business image e.g. bank not like playfulness.
  friendly, helpful, approachable,

Dialog Design Spec (from a caller perspective)

- call flow (inc. error handling)
- prompts
- grammar description
   - names
   - values
- lexicons
- call states and their interface logc
- logging

Grammar - words that are eligible for  recognition

- possible of answers (e.g. operator, none of the options, I would like to ..., .... please
- ROOT: prefix + options + suffix
- few task option, easy distinguisable option,
- open end question (open grammars) (How may I help you) vs listen options
   - open grammars --> Statistical Language Model (SLM)
   - collect sample (5K - 10K)
   - statistical semantic model (SSM)
   - allow flexible system but requires more option

Dictionary - phonetic presenation of how speakers pronounce

- pronunciation dictionary (lexicon)
- can create use dictionary (add or over ride)

Optimization

- caller data --> measure success
- identify problem -
- where to tune? design of the prompt? or technology?
- decide which change is the best.  implement recommended change
- measure the effect of new version
- new products, caller demographic change --> continuous optimizatio

measure of success - 4 area of consideration

- call outcome analysis: how many self-served, how many hangup, how many request agnent, how many request agent without try
- caller experience: caller achived task, caller frustration
- task completion rate: self service rate, individual task completion rate
- recognition performance: grammar cover, prompt to wake, add pronouciation to dict, add confirmation?

The Design of Everyday Things - Donald Norman

Which is best design?

depends on user requirement

good design is user center.

Donald Norman (The design of Everyday Things)

beautiful + useful

- user can figure what to do

- user can tell what is 

going on

Is the company represent the real user (caller)

Good design can delights

Bad design annoys, criminal investigation, loss, complaint can broadcast via social network

bad design costly
good design -- useful and functional 

(identify needs and meet the needs nicely.  function works well)

- anticipates the user's needs (present the solution even before the problem recognized, solve the problem better)

- intuitive (no need to learn)

- result of collaboration

- cost effective (to produce and maintain)

- attractive

Michael Phillips -

We find that the biggest determiner of the success of speech applications is not the performance of the  recognition technology, but rather the match of the user interface to the callers' needs and goals.

task of user interface 

designer

- prompts - init and secondary

- grammars and volcabularies

- call flow (logical branches)

- personality

- global commands (keep caller in control of the flow)

- error handling

- system functions (e.g. barge-in, how long to query DB)

- peripheral concerns (availability of agent, failure scenario..)

design problem

- how familiar callers with the systems

- how much the UI need to instruct callers on how to respond, so that the recognizer can compare the 

utterance with the grammar

- to what extent will logic alone be insufficient

- how can the voice user interface anticipate and approximate what the callers expect

- what social aspects will be important to the design

- what natural and synthetic conversational elements should be included 

Goals of successful UI 

design

- natural, simple and engaging

- honest, builds trust, demonstrates understanding

- quickly, pleasantly and efficiently meets a callers's needs

- is logical, helpful, intelligent

- is predictable, transparent, intuitive

- is responsive when a caller needs -- or does not need -- help

- anticipates possible errors and minimizes those 

possibilities

- cover envetulalities as exhaustively as practical

- educatesc callers on how best to interact with the speech service, allowing them to recover from errors or missteps
- delivers a powerful self-service option for customer care

- communicates and reinforces company brand

design life cycle

6 stages - requirements, design, implementation, testing, deployment, optimization

key deliverables

requirements

- callers , business, application design
- call flow diagrams, sample calls, prototype testing, detailed specification implementation
- prompt coaching and recording, grammar support testing
- usability testing, dialog traversal testing deployment
- pilot tuning - call analysis, UI design updates optimization
- UI and caller updates

Requirement

- foundation of the project
- design demand more input, quality input
- requirement should be documented and reviewed.
- success depends

design team  - get more detail requirement then in sales phase.

- depends on quality of requirement identification

Caller Analysis

1. Who is calling?
2. How/When/From where they are calling?
3. Why are they calling?
4. What do they want to accomplish?

Clear map of caller mental model

- based on user input, client input, observation of user doing the task, call detail records, etc., describe in as much detail as possible how the target user views the task.  In their mind,
- What is the appropriate sequence and number of steps?
- Is there an existing mental model?
- How do they do it now? How flexible is their mental model?
- How complex is the task?

Caller Analysis Worksheet

- different caller types (population)
- for each type: they know / they are looking for

create a small bio for each caller type. --> think their characteristic

1. How technically proficient are callers?
2. option for outbound alerts on due or overdue payments?
3. Should payment information be part of the initial play-out, once the caller is identified?
4. What percentage of callers does each group represent?
5. How frequently do members of each group call?

Define Business Requirement

- business goals
- requirement of business' customers
- corporate image / culture
- existing or legacy services that impact the application (e.g. DTMF IVRS)
- Legal or business constraints
by multiple interview with stakeholder

Define Application Requirements

- tasks caller wants to complete, questions they will ask
- any text-to-speech requirements
- ways to leverage known information for recognition tasks
- ID and verification or other security requirements
- Data available to the application
- Agent availability/restrictions

High-level design and detailed design

High-level design

- define the persona
- interactive, friendly, helpful, credible, trustworthy

Call Flow Design

- think task, think about caller profile
- design with stakeholder which task is important
- show the users' path
- basis of dialog design spec, application development
- not every details (universals, error handling, help routines, user response)