WEBVTT

1
00:00:00.083 --> 00:00:02.585
- What challenges are organizations
running into as they try

2
00:00:02.585 --> 00:00:06.047
to adopt AI inside
their engineering teams?

3
00:00:06.047 --> 00:00:08.508
- One of the key things that
we're seeing is organizations

4
00:00:08.508 --> 00:00:11.052
are rolling out AI tools
for the software engineers

5
00:00:11.052 --> 00:00:13.221
and focusing on adoption.

6
00:00:13.221 --> 00:00:15.473
So are people actually using the tools?

7
00:00:15.473 --> 00:00:17.308
And what we find is
that that's not enough.

8
00:00:17.308 --> 00:00:21.146
Because if you're just using
the tool to answer a question,

9
00:00:21.146 --> 00:00:23.231
it's not actually changing
how you're working.

10
00:00:23.231 --> 00:00:26.067
So the real challenge is
to get software engineers

11
00:00:26.067 --> 00:00:27.610
to fully adopt the tool

12
00:00:27.610 --> 00:00:30.363
and use it in a way that
makes them more productive,

13
00:00:30.363 --> 00:00:33.074
really using the AI to
write code for them instead

14
00:00:33.074 --> 00:00:34.826
of them writing code themselves.

15
00:00:34.826 --> 00:00:37.120
- So Matt, you heard him explain some

16
00:00:37.120 --> 00:00:38.329
of these huge challenges.

17
00:00:38.329 --> 00:00:40.165
You're not just using
the tool to use the tool.

18
00:00:40.165 --> 00:00:41.332
So how did BCG

19
00:00:41.332 --> 00:00:43.460
and AWS come together to partner

20
00:00:43.460 --> 00:00:47.130
to help teams at AWS
overcome these challenges?

21
00:00:47.130 --> 00:00:48.715
- Yeah, well first of all,
I couldn't agree more.

22
00:00:48.715 --> 00:00:51.176
It's not just about the
adoption, it's meaningful usage.

23
00:00:51.176 --> 00:00:54.012
And in our case, that means
are you actually deploying code

24
00:00:54.012 --> 00:00:56.097
to production and is it actually making it

25
00:00:56.097 --> 00:00:57.474
to customers in the form of features?

26
00:00:57.474 --> 00:01:00.310
So the first thing we did is
spent a lot of time upfront

27
00:01:00.310 --> 00:01:01.478
how are we gonna measure this?

28
00:01:01.478 --> 00:01:03.521
Like how are we gonna measure
adoption in the first place,

29
00:01:03.521 --> 00:01:06.441
but how are we gonna measure
that translation from adoption

30
00:01:06.441 --> 00:01:09.319
to POC to production to
actually that meaningful usage?

31
00:01:09.319 --> 00:01:11.905
So we established some
top line metrics together

32
00:01:11.905 --> 00:01:14.157
and then from there we could
set some goals for the project.

33
00:01:14.157 --> 00:01:16.034
Like how are we actually
going to get more code

34
00:01:16.034 --> 00:01:18.703
with AI tools to production
using some common measures

35
00:01:18.703 --> 00:01:20.163
that everyone kind of understands.

36
00:01:20.163 --> 00:01:22.207
- So these common measures
that everyone understands,

37
00:01:22.207 --> 00:01:23.083
I'd like you to touch on

38
00:01:23.083 --> 00:01:24.375
that so that we're very, very clear.

39
00:01:24.375 --> 00:01:28.338
But also what is BCG offering

40
00:01:28.338 --> 00:01:31.132
to help clients improve productivity?

41
00:01:31.132 --> 00:01:32.634
So first you can talk about those tools,

42
00:01:32.634 --> 00:01:36.054
then I wanna get back to you
about what BCG is offering.

43
00:01:36.054 --> 00:01:38.139
- Yeah, well, you know, first
of all it's, it's actually not

44
00:01:38.139 --> 00:01:39.265
as much about the tools.

45
00:01:39.265 --> 00:01:40.850
Like yes you need to
be extra in your tools,

46
00:01:40.850 --> 00:01:42.435
but it's more about the expertise.

47
00:01:42.435 --> 00:01:44.479
And so what we found was hugely beneficial

48
00:01:44.479 --> 00:01:46.272
after we establish what
the metrics are, like

49
00:01:46.272 --> 00:01:49.150
how are we gonna measure
meaningful adoption is embedding

50
00:01:49.150 --> 00:01:52.362
people directly into the
teams who can credibly speak

51
00:01:52.362 --> 00:01:53.530
to using these tools and,

52
00:01:53.530 --> 00:01:55.740
and actually getting to the
outcomes that we both wanted.

53
00:01:55.740 --> 00:01:58.493
So we spent a lot of time,
yes, about measurement,

54
00:01:58.493 --> 00:02:01.579
but then really just embedding
BCG experts into our teams,

55
00:02:01.579 --> 00:02:03.832
earning the trust of our developers so

56
00:02:03.832 --> 00:02:06.126
that they can actually then
see through these projects

57
00:02:06.126 --> 00:02:08.920
and get to more ultimately
production code deployed.

58
00:02:08.920 --> 00:02:11.214
- So you heard 'em talk
about embedding your experts,

59
00:02:11.214 --> 00:02:15.218
but what is BCG doing
specifically to help this process?

60
00:02:15.218 --> 00:02:17.720
- Yeah, so it's actually quite challenging

61
00:02:17.720 --> 00:02:20.557
to get the behavior change to happen.

62
00:02:20.557 --> 00:02:23.434
What we found is there are
actually three reasons why

63
00:02:23.434 --> 00:02:25.019
software developers really resist

64
00:02:25.019 --> 00:02:26.563
using the tools effectively.

65
00:02:26.563 --> 00:02:29.357
One is they don't know that
it can do what it can do.

66
00:02:29.357 --> 00:02:31.526
The second is they have habits,
you know, we're all human,

67
00:02:31.526 --> 00:02:33.111
we do things the way that we do them.

68
00:02:33.111 --> 00:02:35.405
And then the third is there's
this actual real sense

69
00:02:35.405 --> 00:02:36.823
of sort of identity threat.

70
00:02:36.823 --> 00:02:39.826
People worry that if the AI
tool can do what they do,

71
00:02:39.826 --> 00:02:42.203
that it makes them sort of
irrelevant to the world.

72
00:02:42.203 --> 00:02:45.290
And so there's a lot of
sort of fear and resistance.

73
00:02:45.290 --> 00:02:46.875
So what we found is you have

74
00:02:46.875 --> 00:02:48.418
to get over all three of these things.

75
00:02:48.418 --> 00:02:50.837
And so what we do, one
is you do some training.

76
00:02:50.837 --> 00:02:52.922
So you have to tell them
how to use the tools

77
00:02:52.922 --> 00:02:55.008
and have to show them how to
use the tools effectively.

78
00:02:55.008 --> 00:02:58.052
So you know how to actually
explicitly use the tool.

79
00:02:58.052 --> 00:03:00.096
But the second is you have
to give them coaching.

80
00:03:00.096 --> 00:03:03.057
And so we need to have peer engineers and,

81
00:03:03.057 --> 00:03:07.353
and so we will bring in
peer engineers from our team

82
00:03:07.353 --> 00:03:09.063
and then we'll actually with our clients,

83
00:03:09.063 --> 00:03:12.817
help them develop their own
coaches that are peer engineers,

84
00:03:12.817 --> 00:03:14.736
that are experts in using ai.

85
00:03:14.736 --> 00:03:17.113
And those peers actually
coach the developers

86
00:03:17.113 --> 00:03:18.781
and help them develop new habits.

87
00:03:18.781 --> 00:03:20.408
How do I work in a new way?

88
00:03:20.408 --> 00:03:22.535
But maybe even more
importantly, help them realize

89
00:03:22.535 --> 00:03:24.370
that when you use these tools effectively,

90
00:03:24.370 --> 00:03:26.331
it's not a threat to your identity.

91
00:03:26.331 --> 00:03:28.666
It actually makes you much more powerful.

92
00:03:28.666 --> 00:03:29.918
And we find that engineers,

93
00:03:29.918 --> 00:03:33.296
once they have really
adopted a new way of working,

94
00:03:33.296 --> 00:03:35.840
they love working this way,
they love using these tools,

95
00:03:35.840 --> 00:03:37.675
it makes it, their job is more fun.

96
00:03:37.675 --> 00:03:38.927
They don't do toil,

97
00:03:38.927 --> 00:03:41.638
they spend a lot more time
creating and building.

98
00:03:41.638 --> 00:03:42.847
- Right. That's such a key point

99
00:03:42.847 --> 00:03:45.350
because everyone seems threatened by ai,

100
00:03:45.350 --> 00:03:48.228
but what you're saying is
it's not gonna threaten you,

101
00:03:48.228 --> 00:03:50.313
it's going to enhance you.

102
00:03:50.313 --> 00:03:52.232
What about your key goals
for next year, Matt?

103
00:03:52.232 --> 00:03:53.858
What are the key goals?

104
00:03:53.858 --> 00:03:55.735
- Yeah. Well the key goals are,
are really doubling down on

105
00:03:55.735 --> 00:03:57.445
what we've learned like year to date.

106
00:03:57.445 --> 00:03:59.447
And thanks, you know, to the
work we've done with BCG,

107
00:03:59.447 --> 00:04:01.783
we have a 31% productivity improvement

108
00:04:01.783 --> 00:04:04.953
and we've actually shipped
27% more features this year.

109
00:04:04.953 --> 00:04:06.412
And sort of getting back to
what you were saying, like

110
00:04:06.412 --> 00:04:07.747
that's empowering for developers.

111
00:04:07.747 --> 00:04:09.999
They, they went from maybe
some of them, some pockets

112
00:04:09.999 --> 00:04:11.751
of resistance to feeling empowered.

113
00:04:11.751 --> 00:04:13.711
'cause they're shipping
more features, they're,

114
00:04:13.711 --> 00:04:15.713
they're shipping more coded production.

115
00:04:15.713 --> 00:04:17.173
And we wanna, you know,
double down on that end

116
00:04:17.173 --> 00:04:18.800
of the day, what does a developer want?

117
00:04:18.800 --> 00:04:20.677
They want their code in the hands

118
00:04:20.677 --> 00:04:22.178
of customers and customers using that.

119
00:04:22.178 --> 00:04:25.181
And, and they're able to do
that faster and, and do more.

120
00:04:25.181 --> 00:04:27.308
So we're going to do more.

121
00:04:27.308 --> 00:04:29.978
Another thing we're gonna do
is, is work together on not

122
00:04:29.978 --> 00:04:31.437
just the coding part, but other parts

123
00:04:31.437 --> 00:04:32.855
of the software development life cycle.

124
00:04:32.855 --> 00:04:35.108
So, you know, we estimate
that about a third

125
00:04:35.108 --> 00:04:37.485
of the work is coding and
then there's what happens

126
00:04:37.485 --> 00:04:39.779
after you ship the code, like
maintenance and what happens

127
00:04:39.779 --> 00:04:41.614
before all the upfront designs

128
00:04:41.614 --> 00:04:43.866
and specs that go into
what ultimately becomes,

129
00:04:43.866 --> 00:04:45.201
you know, code that you deploy.

130
00:04:45.201 --> 00:04:46.661
And so we're gonna spend a lot of time on

131
00:04:46.661 --> 00:04:49.163
how can we optimize the front
half of that, you know, the,

132
00:04:49.163 --> 00:04:51.624
the specs, the design and
the back half the maintenance

133
00:04:51.624 --> 00:04:53.042
and the ops after you deploy the code.

134
00:04:53.042 --> 00:04:55.712
So the whole software development
life cycle is optimized

135
00:04:55.712 --> 00:04:57.130
and we can move even faster,

136
00:04:57.130 --> 00:04:59.215
Moving even faster.
Thank you so much, gentlemen.

137
00:04:59.215 --> 00:05:00.049
- Thank you.
- Thank you.