BenchmarkInsights

Thevalueofvirtualagenttechnology

ImprovecustomerserviceandboostfinancialresultswithAI-enabledsystems

1

ByJoePetrone,GillianOrrell,and

CarolynHellerBaird

Keytakeways

Significantfinancialbenefits

99%ofstudyrespondentsreportreductionincostpercontactasaresultofusingvirtualagenttechnology(VAT).ArecentForresterConsultingstudyestimatedthatalargeorganizationthatimplementsVATcanachieve$5.50costsavingspercontainedconversation.1Ouranalysisrevealsanaveragecontainmentrateof64%witha38percentage-pointswingfromhighesttolowest.94%ofVATLeadershavealreadyachievedorexceededtheirbusinesscase.

Satisfactionscoresincrease

SomeorganizationsfearVATcouldcausesignificantdecreasesincustomerandemployeesatisfaction.However,wefoundaverageimprovementsof8and7percentagepointsincustomerandagentsatisfaction,respectively.Averagehumanagenthandletimedroppedby12%.VATLeadersachieveevenmore:averageimprovementsof12and9percentagepointsincustomerandagentsatisfactionanda15%reductioninhandletime.

Leadersimplementedearly,integratedVATwithbackendsystems,andtraineditonhighvolumesofcontacts.

Risingtothechallenge

WhenCOVID-19struck,organizationswithmatureVATcapabilitiesrapidlyexpandedoperationstosupportbusinesscontinuity.Themajorityoftheseorganizationssawanincreaseincustomersatisfaction.

Farbeyondexperimentation

Virtualagenttechnology(VAT)ishavingitsmoment.Ifyou’verecentlycalledacompany’scustomercarenumberorconductedanonlinechataboutaservice,product,orpayment,thereisagoodchanceyouwereinteractingwithvirtualagenttechnology.VAT’spopularityissurgingbecauseofthetechnology’sseemingabilitytodoeverything:betterservecustomers,enhanceemployeeexperience,reducecosts,andincreaserevenue.

TheCOVID-19pandemichasrocketedVATadoption

intohyperdrive.Foremergencysituations,whenspeedandsimplicityaremissioncritical,out-of-the-boximplementationscangoliveinamatterofhours.However,companiesthatthoughtfullydesign,develop,anddeployVAThavethepotentialtounlockevenmorevaluefortheirorganizationsandtheirusers,betheycustomers,employees,orhumanagents.

Untilnow,actualdataaboutVATperformancehasbeenlimitedtoindividualcasestudies,makingitdifficultfororganizationstoderivemeaningfulcomparisons.Asweembarkedonthisresearch,wekneworganizationsneededanswerstofourkeyquestions:

–HowefficientisAI-enabledVAT?

–WhatarethekeyattributesandpracticesthatdrivevaluefromVAT?

–WhatistheimpactofAI-enabledVAToncustomers,endusers,andhumanagents?

–WhatimpactdoesVAThaveonorganizations’financialperformance?

MeasuringVATperformance

VATperformanceismeasuredbyacombinationofnewmetrics,uniquetothewaythetechnologyworks(seeFigure1),andtraditionalcontactcentermetricsimpactedbythetechnology.

1.Intentrecognition

Intentisthecustomer’spurposewhenmakingcontactandduringthecontact,forexample,questioningachargeoraskingaboutaproduct.Intentisdistinctfromcontactbecausecustomersmayhavemorethanoneintentpercontact.VATistrainedtorecognizeandrespondtoeachintentseparately,makingitvastlymoresophisticatedthanatraditionalkeywordsearch.

Customersmightsay,“HowdoIsettlemyaccount?”whentheirintent—fromVAT’sperspective—is“Paymybill.”TrainingVATtorecognizetruecustomerintentbehindthemanywayscustomersexpresstheirintentiscriticalforsuccessfulimplementationandmaintenance.TheskillrequiredtoprogramVATtoaskgoodclarificationquestionsshouldnotbeunderestimated(see“Insight:Conversationaluserexperience(UX)design”onpage4).

AverageintentrecognitionforallsurveyrespondentsacrossallthreetypesofVATimplementationsis68%.ForsituationsinwhichVATdirectlyinteractswithexternalcustomers,theaverageis70%.Andforintelligentagentassistimplementations,theaverageintentrecognitionis65%.ThiscouldbeareflectionoforganizationsmakinglargerinvestmentsinVATsystemsaimedatexternalcustomers.

Figure1

Thenewcontactcenterperformancemeasures

VATdirectlyinteractingwithcustomerorenduser

1Intentrecognition

2In-scopesegment

3Containment

DoestheVATunderstandthecustomer’sneeds?

IstheVATtrainedtomeetthecustomer’sneeds?

DoestheVATmeetallthecustomerneeds?

68%Median

75%Benchmark

Inboundcontacts

Humanagentsupportedbyagent-assistVAT

Intentrecognition

65%Median75%Benchmark

2In-scopesegment

62%Median70%Benchmark

64%Median70%Benchmark

64%Median70%Benchmark

Humanagentrouted

1

Source:IBMInstituteforBusinessValue.3

4

2.In-scopesegment

Oncevirtualagenttechnologyrecognizescustomerintent,itshouldalsoknowifithasbeentrainedtosupportthatintent.Acrossallsurveyrespondents,theaverageproportionofinboundcontactswithintentsthatfallwithinthescopeoftheVATis63%.Thedifferencebetweenthelowestandhighestin-scopesegmentis36percentagepoints.

3.Containment

Notallcompaniesmeasurecontainmentthesameway.SomedefinecontainmentasthepercentageofallcontgctsfullyhandledbytheVAT.ThiscanresultinlowcontainmentpercentageswhentheVATdoesnotyet—orwillnever—handlecertaintypesofcontact.

Inthisstudy,containmentistheportionoftotalcontactstheVAThasbeentrainedtohandlethatitresolveswithoutanyescalationto—orinvolvementfrom—ahumanagent.

Measuringcontainmentatcontactlevelispragmatic,thoughitcanunder-reporttheachievementofVATwhensome—butnotall—ofacustomer’sintentsareresolvedbyVAT.Theuseofsophisticatedandautomatedanalyticstomeasurecontainmentfairlyisimportantforfuturedevelopment.

Averagecontainmentacrossrelevantrespondentsis64%Thedifferencebetweenthelowestandhighestreportedcontainmentis38percentagepoints.Thisisawiderange,andthefinancialimplicationscanbesignificant.EverycontactcontainedbyVATmeansnohumanagenttime.BasedonIBVanalysisofhumanhandletimeacrossallcontacts,anaverageoffourminutesofhumanagenthandletimeissavedwhenacontactiscontainedbyVAT.

Performanceforcontainmentisstronglylinkedwithmachinelearningandsystemsintegrationfactors(see“NorthAmericangrocerychain:OutsourcingtoAI”onpage6).Forexample,66%ofrespondentsinthetopquartileforcontainmentreportthattheirVATcanexecutecomplexactionsonbehalfofcustomers,comparedwithonly23%inthebottomquartile.

Insight:Whatwemean

byvirtualagenttechnology

Virtualagenttechnologyreferstothedeploymentofautomationthatusesacombinationofartificialintelligence(AI)technologiesorcapabilitieslikemachinelearning,naturallanguageprocessing,naturallanguagegeneration,sentimentanalysis,languagetranslation,speech-to-text,roboticprocessautomation(RPA),andopticalcharacterrecognition(OCR)toautomatedialogue,andinsomecasesbackendprocesssteps,toserveendusers.

–Thismayormaynottaketheformofachatbot.SomechatbotsqualifyasVATbecausetheyincorporatemachinelearningaswellasnaturallanguageprocessing.

–Integratedvoiceresponse(IVR)systemsonlyqualifyasVATwhentheyuseAItopoweraconversationalexperienceandtheuserisnotlimitedtoutteringasetofspecifickeywords.

Insight:Conversationaluser

experience(UX)design

Conversationaluserexperience(UX)isanascentareawithindesignthatfocusesoncreatingsystemsthatcanengageinnaturalconversation.Userinteractionworksthroughthewordsalone,whethervoiceortext.Thisrequiresformalknowledge,adaptedfromsocialscience,ofhowhumanconversationworks.

Whilesomechatbotsandvoiceassistantsarelimitedtoansweringquestionsorfollowingcommands,virtualagenttechnologyusingnaturalconversationcanengageinawiderrangeofactivities,fromhellostogoodbyes.Thegoalismutualunderstanding,anditincludesmanynaturalmechanicsfordisplayingandrepairingmisunderstandings.Ontopofthisfoundation,designerscancreateexperiencesthatreflecttheorganization’sbrandqualities.3

5

Machinelearningfactors

Organizationalfactors

Durationinuse

DegreeofintegrationbetweenVATandothersystems

Numberofseparateintents

inscope

%ofcontactsbeinghandled

Volumeofcontactsbeinghandled

ComplexityofactiontheVATisenabledtotake

InfluencingVATperformance

ManyfactorshavethepotentialtoinfluenceVATperformance,fromlevelsofstrategicintentandinvestmenttotheskillsofthepeoplemaintainingitandthechoiceofspecifictechnologiesimplemented,suchasthestandardofparsersusedfornaturallanguageprocessing.AlargepharmacystorechainintheUSimplementedsentimentanalysisforcontactsthatareescalatedbytheVATtohumanagentssoagentscanquicklyprioritizecustomersatriskofbecomingdissatisfied.Thisenablesagentstodirectlyinfluencetheirmostimportantmetric:customersatisfaction.

Figure2

FactorsinfluencingVATperformance

Somefactorsinfluenceindividualmetricsandothershavethepotentialtoinfluencebroaderperformanceacrossmultiplemetrics(seeFigure2).Machinelearningandsystemsintegrationfactorshavethegreatestimpactonlevelsofrelativeperformanceandareinterrelated.

Machinelearningfactors:ThelongertheVATisinuse,themorecontentithasbeentrainedtohandle,andthemoreitdoes,thebetteritshouldperform.

Systemsintegrationfactors:ThegreaterthelevelofintegrationbetweentheVATandothersystemsandthemorecomplexactionstheVATisenabledtotake,thebettertheresults.

Organizationalfactors:Specificcircumstancessuchastheemphasisplacedontrackingresultsandindustry-relatedaspectssuchascomplexityofcustomercontactsanddigitaluptakecaninfluenceperformance.

Systemsintegrationfactors

Type(s)ofVATimplemented

LevelofinvestmentinVAT

Performancefocus

Industry

6

Leaderattributes

Ouranalysisidentifiedasmallgroup(105organizations)ofVATpioneerswecallthe“Leaders.”EachLeaderdemonstratesthreecriticalattributesthataretheresultofdeliberatebusinessdecisions:

–EarlyadopterofVAT

–LargeproportionofinboundcontactsarewithinthescopeoftheirVAT

–HighdegreeofintegrationexistsbetweentheirVATandbackendsystems.

Comprising10%ofoursample,Leadersspanallindustriesandregionssurveyed,aswellasallthreetypesofVATimplementation,inalmostequalproportions.Leadersalsoreportoutperformingtheircompetitiononrevenueandprofitabilityforthelastthreeyears.LeadersoutperformotherrespondentsoncoremetricsusedtomeasuretheefficiencyofVAT(seeFigure3).

Figure3

TheefficiencyofLeaders’VATfarexceedswhatothersreport

80%

66%

75%

62%

74%

63%

Firstcontactresolution

Intent

Containment*

recognition

LeadersOthers

*Containmentdefinitionforthisstudy:Theportionoftotalcontactsthatthevirtualagenttechnologyhasbeentrainedtohandlethatitresolveswithoutanyescalationtoorinvolvementfromahumanagent.

NorthAmericangrocerychain:OutsourcingtoAI

Competitionamonggrocerychainsisfierce,andwiththinmargins,customerloyaltyiscritical.AlargeNorthAmericangrocerychainrecognizedthatasmartcustomercontactcentercouldenableastronger,expandedloyaltyprogram.Hiring,training,andretainingcallcenteragentsischallenging.Thechainneededacost-effective,predictable,scalablewaytohandleincreasingcallvolumesandcomplexitywithminimalcapitalinvestment.

ThecompanyturnedtoIBMtodesign,build,andoperateanenterprisecognitivecallcentersolutionthatleveragesVATwithvoiceintegration.Thesolutionfeaturescallflowsdesignedtooptimizecustomerexperience,whileloweringoperationalcostsandscalingforthefuture.

Integratedwithbackendsystems,theVATsystemisenabledtoauthenticatecallersandtakeaction.Eachcallisloggedinthecompany’scustomerrelationshipmanagementsystem,whichcantrackthesolution’sperformancelikeitwouldforahumanagent.Thechainonlypaysforcallssuccessfullyhandledandsaves40%oneachcontainedcall.It’slikeoutsourcingthecallcentertoVAT.

7

“Yourfirstreleasewillbeyourworst.”

Virtualagenttechnologydeveloper,financialservicesindustry,UK

Continuousimprovement

ContinuousimprovementisacriticalcomponentofVATperformance.ReviewingandretrainingVATisconductedbyskilledhumansandthroughautomation.Whileallrespondentsreportusingautomation,moreoftherespondentsoutsidetheLeadergroupusemanualmethods(41%)thanLeaders(33%).

Leadersdon’tjustpassivelymeasure;theyact.NearlyallLeadershaveexpandedthescopeoftheirVATinresponsetotheirreviews,comparedwith59%ofothers.WhetherenhancementsentailadditionalVATtrainingorupgradestotheVATinterface,Leadersaretakingactiontocontinuallyimproveperformance(seeFigure5).

Butit’sthewayLeadersselectvirtualagentconversations

forreviewthattrulydifferentiatesthem.Leadersuseawider

rangeoftechniquesandmoresophisticatedmethods

(seeFigure4).

Figure4

Howvirtualagentconversationsareselectedforreview

Customerintent

86% 68%

Sentimentortoneanalysisresults

74% 55%

ContactswhereSLAsmissed

74% 51%

Neurolinguisticprogramconfidencelevel

64%

14%

Specificcallcategories,e.g.,abandonedcallsorbyagent

58%

51%

Highvolume,lowcontainment

58%

Figure5

Resultsofvirtualagentqualityreviewsinthelast30days

Scopeofvirtualagentexpanded

91% 59%

Additionalvirtualagenttraining

82%

64%

Customerinterfacechanged

82%

64%

Scopeofvirtualagentreduced

2%

13%

LeadersOthers

45%

Algorithm

55%

36%

Random

30%

49%

LeadersOthers

8

Most

VATimpact

Virtualagenttechnologydirectlyimpactscustomers,humanagents,andorganizationfinancialresults.

Impactoncustomers

ThereareavarietyofreasonswhyorganizationsadoptVAT,butimprovingcustomerexperienceis,byfar,themostcitedone(see“HongKongbank:ContinualVATenhancementpaysoff”).For46%ofLeadersand33%ofotherrespondents,itisthesinglemostimportantdriver(seeFigure6).

Figure6

ReasonsforimplementingVAT

important

reason

All

reasons

Leaders

Others

Leaders

Others

46%

22%

8%

13%

11%

33%

15%

10%

19%

23%

88%69%68%48%41%

72%45%42%61%

64%

CustomerexperienceReducecost

ImprovebranddifferentiationIncreaserevenue

Improvehumanagentexperience

HongKongbank:ContinualVATenhancementpaysoff

Inthequesttoimprovecustomerutilityandeaseofengagement,aleadingbankinHongKonglaunchedafirst-of-its-kindAIvirtualassistanttoaddressretailcustomers’inquiresforselectservices.Overthreeyears,theVATwasextendedtosupport13services,includingaccountservices,paymentplatform,billpaymentservices,andcreditcard,withcontextualconversationinChinese(CantoneseandSimplifiedChinese),English,andevenamixoflanguages.

Today,thecompany’sVATcantalkwithacustomertohandlepersonalbankingrequests,suchasamoneytransfer,andisintegratedwiththebank’se-bankingservices.TheVATsystemhaswonmorethan15awards,andcustomeruseofthechannelhasgrownbyafactorof6.

ThissuccessledthebanktoimplementVATforitsbusinessbankingdigitalengagementplatform.Here,too,resultsareimpressive:Usersurveysreport75%satisfaction,anduseoftheVATchannelhasgrownbyafactorof10.

9

99%ofallrespondentsreportpositiveabsolutechange-to-dateincustomersatisfactionforallinboundcontactsasaresultofusingVAT.Inotherwords,regardlessofwhetherimprovingcustomerexperiencewasthedriverfortheirimplementation,practicallyallreportanincreaseincustomersatisfactionscoresasaresultofusingVAT.ThevastmajorityofLeaderscreditVATforhavinga“significantlypositive”impactontheircustomersatisfactionscores(seeFigure7).

Figure7

HowtheuseofVAThasimpactedcustomer

satisfactionscores

12

8

70%

63%

Customersatisfactionlast30days

Percentage-pointchangeincustomersatisfactionfromusingVAT

LeadersOthers

ImpactofVAToncustomersatisfaction

1%

2%

Leaders

20%

12%

25%

Others

67%

73%

SomewhatnegativeimpactSomewhatpositiveimpact

NeutralSignificantlypositiveimpact

Vodafone:VATsuccess

Aspartofitscustomerexperiencestrategyanddigital

transformation,VodafoneGroupbecameoneofthe

world’sfirsttelecommunicationscompaniestoadopt

agenttechnologyforcustomerservice.Today,TOBi,

Vodafone’sAI-baseddigitalassistant,isavailable

throughouttheUK,Europe,Africa,andtheMiddleEast.

TOBiprovidesacentral,modularplatformtowhich

channelscanconnect,enablingindividualmarketsto

prioritizethechannelsthatsuitthepreferencesand

needsoftheircustomers.Channelsdeployedinclude

SMS,theMyVodafoneapp,localVodafonehomepages,

Facebook,WhatsApp,andAmazonAlexa.

Asaresult,thenumberofVodafonecustomers

messagingversuscallingthecallcenterhasalmost

doubled.InItaly,two-thirdsofcustomercontacts

arenowcontainedthroughTOBi.

10

NPS

points

Revenue

increase

▲5pts

▲3%

-15%▼

-18%▼

Humanagenthandletime

*Percentage-pointincrease

Impactonhumanagents

Agentswhofeelvaluedandempoweredwiththepropertoolsandsupportaremorelikelytodeliverabetterexperiencetocustomers.Andthecostofreplacingadissatisfiedagentwholeavestheorganizationcanbeasmuchas33%oftheexitingemployee’ssalary.4

Leaders,especially,arekeentoimprovethehumanagentexperience;68%citeitamongtheirreasonsforadoptingVAT,andourdatasuggestsitishavingapositiveimpact.Bycalculatingallrespondents’reportedchange-to-dateasapercentageoftheircurrenthumanagentsatisfaction,wedeterminedVAT’saveragecontributiontocurrenthumanagentsatisfactiontobeanimpressive20%.

VATalsoimpactshumanagenthandletime.Averagechange-to-dateasaresultofusingVATisa12%reduction,withLeadersreportinganaveragereductionof15%.

Theonlyareawherestudyresultspointtowardaminoradverseimpactishumanagentattrition.Absolutechange-to-datewasanaverage1%increaseinannualagentvoluntaryturnover.Inthecontextoftypicallevelsofcontactcenterturnover,thisisnotlarge,butthedirectionisinteresting.ThisfindingcouldsuggestthatVATisafactorinthedecisiontoleaveforsomeagents,whileitcontributestoimprovedsatisfactionforthosewhostay.

Impactonfinancialperformance

Whenyoureducethetimeittakeshumanagentstoresolvecontacts,youreducecosttoserve.Whetheryou’veempoweredagentswithVATtoreducetheirmanualeffortoryou’veprovidedcustomersoremployeeswithVATforself-serviceortoconducttheinitialintercept,theseimplementationscanhaveasignificantfinancialimpact.

Forexample,theForresterConsultingstudyestimatedthatalargeorganizationcouldachieveanaveragecostsavingof$5.50percontainedconversationusingIBM’sWatsonAssistant?.AconversationalAIplatform,WatsonAssistantallowsuserstointeractwithbusinesssystems

usingnaturalhumanlanguage.ForresterConsultingestimatesthatoverthreeyears,withaconservative25%containmentrate,thecostsavingswouldbeworthmorethan$13million.5

Respondentsinourstudyreportbothtop-andbottom-linebenefitsfromusingVAT.EveryrespondentindicatesthatVAThascontributedtoanincreaseinorganizationrevenue,theaverageincreasebeing3%.TotalrevenueupliftforrespondentsreportingresultsforVATdirectlyinteractingwithexternalcustomerssumsto$5billion.6

Inaddition,99%ofrespondentsreportthatVAThasreducedtheircostpercontact,with25%citingareductionof20%orgreater.

Leaders,particularly,haveenjoyedVAT’sbenefits,boththeincreasedsatisfactionanddecreasedcosts(seeFigure8).

Figure8

Leaders’averagereportedimpactofusingVAT

Firstcontactresolution*

Customer

satisfaction

▲14

increase*

▲12

-18%▼

Costpercontact

FTEshandlinginboundcontacts

11

Leaders

6%

64%

5%46%

42%

68%

13%

56%

44%

19%

Returnoninvestment

54%ofallrespondentsreporthavingalreadyachievedorexceededtheirexpectedreturnoninvestment(ROI)fortheirVATimplementation.94%ofLeadersareinthispositioncomparedto49%ofothers(seeFigure9).Ittook,onaverage,twoyearstoachieveexpectedROI.ForthosewhoexceededROIexpectations,theaveragetime-in-usewasonlysixmonthslongerattwo-and-a-halfyears.

64%ofallrespondentsuseathresholdrateofreturnfortheirVATdecision,whileclosetothreequartersofLeadersdo.TheaveragethresholdrateofreturnforLeadersis25%,nearlydoubletheaverageofotherrespondents.

Figure9

AchievementofROIforVAT

OptimizingVATinvestments

Organizationsaretransformingtheirinfrastructures,platforms,andcloudstrategiesalongsidetheirVATinvestments.VATmaturityforcustomercarereliesheavilyonanorganization’sabilitytolinkandleveragetheever-increasingvolumeofcustomerdataacrossinternalandexternalsources.Cloudisakeyenablertoachievethisacrosschannelsandecosystems.

Morethan200surveyrespondentsprovideinsightsintotheirorganizations’useofcloudcomputingacrosstheirservicedesksystems.Allreportusingcloudforcustomerrelationshipmanagement(CRM)andchatmessagingsystems,aswellastheirVAT.ConsiderablymoreLeadersreporthavingfullyintegratedcloudstrategiesfortheirservicedesksystems(seeFigure10).

Figure10

Strategytomoveservicedesksystemstothecloud

30%

Others

7%

HavenotachievedROIanduncertainifitwillbeachievedHavenotachievedROIbutexpecttodoso

HaveachievedexpectedROI

HaveexceededexpectedROI

LeadersOthers

Fullyintegratedcloudstrategy

forservicedesksystems,withspecificactionsplansandKPIs

Cloudstrategyforservicedesk

systemswithspecificactionplansandKPIs

Cloudstrategyforservicedesk

systems,butitlacksspecificactionplansandKPIs

12

9%

44%

47%

49%

“ItwasCOVIDthatsparkedVATasanopportunity.

Itdroveustoinnovateandtakesomerisksto

quicklyimplementsomethingwe’dalreadyplanned.

Thefasteryoulaunchsomething,testit,anditerate

onit,themorevalueyoucangetoutofit.”