SAM RANSBOTHAM: When outcomes don't
motivate artificial intelligence efforts,
how can they be successful?
Find out how one chief data officer thinks about AI
on today's episode.
JACK BERKOWITZ: I'm Jack Berkowitz from ADP,
and you're listening to Me, Myself, and AI.
SAM RANSBOTHAM: Welcome to Me, Myself, and AI,
a podcast on artificial intelligence in business.
Each episode, we introduce you to someone innovating with AI.
I'm Sam Ransbotham, professor of analytics at Boston College.
I'm also the AI and business strategy guest editor
at MIT Sloan Management Review.
SHERVIN KHODABANDEH: And I'm Shervin Khodabandeh,
senior partner with BCG, and I colead BCG's AI practice
in North America.
Together, MIT SMR and BCG have been researching and publishing
on AI for six years, interviewing hundreds
of practitioners and surveying thousands
of companies on what it takes to build and to deploy and scale
AI capabilities and really transform
the way organizations operate.
SAM RANSBOTHAM: Today, Shervin and I
are excited to have Jack Berkowitz, chief data
officer at ADP.
Jack, thanks for joining us.
Welcome.
JACK BERKOWITZ: Thank you.
Glad to be here.
SAM RANSBOTHAM: Let's get started.
You're the chief data officer at ADP.
Can you tell us about what that role means?
JACK BERKOWITZ: ADP, known as Automatic Data Processing,
is the world's largest provider of HR services, payroll, taxes,
things like that.
We operate in 140 countries.
We have over 900,000 clients.
Millions of people are getting paid from us every day.
I sort of have a two-sided job.
On the one hand, I'm responsible for all of the data that
flows through our systems.
We're a really big company.
We have massive amounts of data, so [it involves] all the things
that are classically associated with chief data officers --
things about data governance, data security,
usage of analytics.
The other side of my job -- and it's probably even a bigger job
-- is I build data products, and so my team builds people
analytics, benchmarks, compensation information,
all [those] types of products that our clients are using
to take decisions about the world of work every day.
SAM RANSBOTHAM: I didn't hear the words artificial
intelligence in there anywhere.
How is that involved?
JACK BERKOWITZ: I also run that for the company as well,
but we use machine learning throughout those processes --
whether we're cleaning the information,
whether we're building embedded capabilities in our HR
applications or our payroll applications,
whether we're doing things like aligning job titles.
People would say, "Well, how hard can that be?"
You know, in any given month, we pay about 21 million people.
We have about 14 million job titles,
and we crunch that down to between 6,000 and 8,000 job
titles -- so [there's] an awful lot of very sophisticated
natural language processing and machine learning to make that
happen.
SHERVIN KHODABANDEH: It seems like there's
three different roles that you mentioned
that all come together.
And I say this because at many companies,
there are literally three different roles for what you
mentioned -- for data governance, for data products,
and for AI -- which creates maybe a bit of siloed-ness
and a bit of maybe disconnectedness,
because all these things have to work together.
Comment a bit, please, on how it came about that it's
one person leading all three.
That's my first question.
And then, my second question is, is the AI involvement
only in the data products, or is it
a broader role that you have that you're also supporting
AI for the broader enterprise?
JACK BERKOWITZ: It's a really good question.
The thing to know about ADP is, yes, we're a services company,
in the sense that we provide, for example,
payroll for about 1 in 6, or even more than that,
people in the U.S. But we also are a SaaS [software
as a service] product company, and because of that,
there's a whole bunch of different development
organizations working on building SaaS products,
whether it's for the small businesses
all the way up to the biggest companies in the world using
our applications to do HR or recruiting or payroll
or taxes, things like that.
And because of that, this role emerged, really.
It started as building data products,
but to build data products and things like reporting,
it grew the data platforms.
And off the data platforms, it grew more and more capabilities
in terms of doing machine learning, best practices.
We got into the ethical use of data
and the ethical use of machine learning and AI,
and that allowed us to be additive in terms
of capabilities.
The other thing about it, then, is, OK,
well, where's the extent?
Because we have all of those SaaS applications,
my teams will sometimes build the embedded capabilities
for other applications.
But we also enable those other development organizations
to use the frameworks that we build.
We, for example, build a whole bunch of machine learning
operations capabilities -- things about bias monitoring
and data shape monitoring -- because that makes sense to be
done once in a company and then allow other people to take
advantage of it.
We've seen a massive growth in people identifying themselves
as data scientists over the past four years.
We've been hiring people and everything else,
but they don't all have to learn how to do model
deployment into production.
SHERVIN KHODABANDEH: Very interesting.
Is it fair to say that introduction and maybe
scaling of AI more broadly outside of the data products
that you do for your customers was sort of the data
products themselves?
The incubation of these data products
opened the eye of the organization.
JACK BERKOWITZ: Yeah, exactly.
It was exactly that.
It was this incubation.
And then, off the incubation, we started
to see areas of opportunity and areas of excitement.
It wasn't really a top-down push.
It was very much a bottom-up, where
teams were seeing what we were achieving,
and then other teams would come to us and say, "Hey,
wait a second.
We want to build a capability.
Can you work with us?"
And so it's really become an organic growth.
SHERVIN KHODABANDEH: I really love this story.
Often, I get asked to talk to groups or [do] interviews
with media around chief data officer roles,
and there's a question around, "What's the right chief data
officer role?"
And I've always been saying that role has to be really,
really linked to the use of data,
not just to the governance of data
and to building things with data.
And I think you're a great example
of the right setup of that role and success with that role.
JACK BERKOWITZ: It's interesting,
because my career's all been about product development
or outcomes.
It's been about making sure that you have business outcomes.
Are you building something, are people buying it,
or are you building something and you're coming out
with a better capability?
We bring that product mindset to even our data governance.
Yes, we need to do governance, but it's not
for regulatory compliance, necessarily.
It's really about making sure that we understand
the information such that somebody can build a good data
product on top or a good machine learning capability on top.
Otherwise, why are we doing all this?
I graduated right at the time of a recession.
Sound familiar?
I spent about 10 years in engineering consulting,
mostly for DARPA, which is the Defense Advanced Research
Projects Agency, which gets you involved in interesting things.
From there, I decided to start a company with a few friends
and got into the startup world, and then I
had a great opportunity to join Oracle, maybe 11 years ago now,
and really enjoyed my time there.
And then I was able to bring it to ADP four years ago.
ADP has really been the pinnacle of my career.
I couldn't have asked for a better situation in terms
of combining all those learnings of how you watch a user doing
something, how you start a business
in these little startup companies that were VC-backed,
to the data and the technology.
We have to run these systems 24/7.
Companies depend on these systems
to pay their employees, which is, one would argue,
one of the most important things that exists in a company,
particularly today, in today's environment.
SHERVIN KHODABANDEH: Can you share with us
some uses of AI in the products that you
build for your customers, as well as
maybe those that are broadly used for the enterprise
or for maybe core processes or more internally focused?
JACK BERKOWITZ: We're in the HR space,
so [that] runs a wide range of capabilities.
One of the things that we're doing right now [that]
we're really excited about is, we've used that job title
information along with a lot of other natural language
processing to come up with a skills graph, --
a 100% data-driven skills graph.
A lot of other vendors do it with hand-cranked ontologies.
Downstream from that, that skills graph
shows up in a variety of places, whether it's
in recruiting applications, the employee profile
inside of a company so that people can find
new roles, things like that.
We have a group that's doing recommendations for people
in retirement programs.
We have a big retirement program for people in small businesses.
A lot of times, [with] small business, people
aren't offered healthcare or retirement.
ADP makes those services available to small-business
people to offer to their employees.
And there's a capability that recommends to people
to say, "Hey, people like you will actually
invest more or less in their retirement program."
And so that's a machine learning-based capability.
But then we also, just like any other company,
use machine learning across the board in other areas.
We do a lot of things in our sales and marketing channels.
But, more importantly, we do a lot
of things in our service [channels].
So we're doing an awful lot right now
to create a self-service environment for our clients.
Our ability to create a better service environment for them
creates a better experience, right?
They get more accurate pay or they get a better experience
for their employees.
In turn, that's better business for us,
so we're spending as much money and as much time
on making that service experience great
as we are [on] making the core product great.
SAM RANSBOTHAM: I know you've advocated for the idea of an AI
ethics board.
I'll take the counter: Why is that important?
What's the benefit of that?
Why bother setting up AI ethics boards?
JACK BERKOWITZ: We started it because we just
felt that the pace of technology and the pace of data
probably weren't representing the values that we wanted
to represent with our clients.
We started it originally because we thought
it was the right thing to do.
Where it's gone from there has really been interesting.
We're learning a heck of a lot, both in terms
of our own product development but also externally
about how to educate not just our clients but even
our ADP associates, [and] in terms of how we evaluate where
we want to do business, like biometrics or voice
recognition, or even what data access rights mean.
There's also now, three years later,
a big regulatory push, both in the EU,
the FTC of the United States, the EEOC [U.S. Equal Employment
Opportunity Commission], and so we're not reactive to that.
We know what to think about.
We're in a great position to deal with it
by thinking a little bit ahead.
SHERVIN KHODABANDEH: From a setup and accountability
perspective, do you see the topic
of ethics and responsible AI governed by a board
or governed by a person advised by a board?
JACK BERKOWITZ: We're much more in the latter,
and we want to do that for a reason.
We bring external experts onto our board.
We have people from the HR domain.
We'll have people from the machine learning world.
We'll bring [in] ethicists.
We want that board to have freedom of thought.
We have very structured product release processes,
whether it's for the products that we release to clients
or whether it's the products that we use internally.
And we have governance throughout there, and security;
if you can imagine, data security
is at the top of our list at the moment.
Also on the board is our chief privacy officer.
So we want the board to have freedom of thought.
And it's an adviser.
Project teams have to present to the board
as part of going to market for areas.
SAM RANSBOTHAM: It's interesting, Shervin.
You know, Jack, I don't want to discount
how important and sensitive your data is,
because clearly it's very sensitive data.
But it's interesting.
Shervin and I talk with people; we'll
talk with people in medical and health,
and everyone seems to have this moment of, like, "Oh gosh;
oh, our data is really sensitive and important."
And I think maybe that's ubiquitous now,
that all data seems to be like that.
I guess a lot of people can learn from the board set up
like this.
JACK BERKOWITZ: It's not a bad thing
to be protecting people's interest in their information.
SAM RANSBOTHAM: It's really pretty fascinating,
because there are lots of sources of information
about salary, and people self-report in lots of areas,
but you've got ground truths on a lot of information about what
actually hits their bank accounts.
It gives great insight into really
what's going on in the economy.
JACK BERKOWITZ: It creates a unique capability,
both to be able to provide that information to our clients
or to their employees or associates,
but also to treat it properly, right?
We have a great opportunity to treat it properly.
And so all the levels of data security,
all the levels of all the great things CDOs care about --
you know, data governance, providence, lineage --
we have a wonderful opportunity to practice the field.
SAM RANSBOTHAM: One area that I think you probably
are interested in mentioning is this idea of comparing
DEI [diversity, equity, and inclusion] metrics.
That's a great place that you've been
able to provide benchmarking and give insight into what's really
going on versus what people would
like you to think is going on.
JACK BERKOWITZ: Yeah.
It's a great point.
The company actually started that in 2017, [when it]
published its first pay equity explorer,
which allowed companies to take a look at how
they were doing in terms of pay equity
gaps for disadvantaged groups.
Now we have the benchmarking capability that
allows a company to see, for their location,
for their industry, for their company size,
how are they doing in terms of creating a diverse environment,
and then also, how are they doing not just
bringing people in, but actually advancing them
in their careers?
By bringing all that together, by using our benchmarking
capability, by solving a problem,
by looking at an outcome, we've had great success.
We can run multiregression analysis down to the individual
-- not just inside their company,
but against the diverse population in that local
geography or industry.
But then we can say, "OK, here are four or five budget
scenarios."
Because it's one thing to say, "Hey,
you have pay equity issues."
But, you know, maybe the company has budgetary things,
so they can make some choices about budgetary scenarios,
and it tells them, "OK, if you want to close this budget,
these are the people that you're able to cover."
And so they basically can change that.
They come out and then, boom, they
can make those changes straight into people's paychecks.
And it's a meaningful impact.
SHERVIN KHODABANDEH: This is wonderful.
The question I have, which would be
very interesting for our audience,
is, where do you get started?
Because we see ...
I mean, I see this in my work, [I] see this with Sam when
we interview and research the topic of AI deployment,
there's a question around, how much do you build capabilities
before you begin to monetize or commercialize or build data
products or use cases, versus how much value-
and use case-driven you are.
And I'm really interested in your perspective,
both for ADP and any advice you have
for others in the early stages of their journey.
JACK BERKOWITZ: The way I've always
looked at it is, if you're building a product,
whether you're a startup company or any other,
is, build the thread from one corner of the piece of paper
up to the other corner of the piece of paper.
And use that thread -- in other words, a use case or two --
to help you define what you need in your situation with
your company at that time.
You could say those are prototypes, but in my mind,
a prototype is useless unless you actually
try to have an impact with it, because you don't learn
about how to measure outcome.
You won't learn how to measure what you actually need.
The thing that's been lost in machine learning and all
the buzz over the past six, seven years
is that all machine learning and all AI
is targeted to an outcome.
To me, it's really about fielding some capability.
Off of fielding that capability, you'll
learn what levels of machine learning operations you need,
you'll learn what levels of data you need.
And I know that there's 1,400 vendors.
Matt Turck has the great FirstMark Capital matrixof it,
and I remember when that was only 30 vendors, by the way.
I know all 1,400 vendors will tell you
[that] you need to buy their stuff right away,
and that's just not true.
That's just not true.
You've got to buy some of it, though,
in order to get that initial thing fielded.
SHERVIN KHODABANDEH: Yeah, I'm so
happy you say that, because that's, honestly,
been the unlock we've seen both in our research, Sam, where we
see that firms that just take technology first maybe
get a little bit of value but there's
a big piece they don't get.
But also, in our work at BCG, that's
been the major unlock, to be value-driven and
outcome-focused.
And I like how you talk about the thread,
because you cannot just build these things in silos,
but it doesn't have to mean that you build the full stadium
before the baseball [game] begins.
You could start playing.
JACK BERKOWITZ: Exactly.
At a company before ADP, we used to call it
a "Field of Dreams business plan."
It's like, "Nobody ever invented baseball; why are we
building a baseball field now?"
The whole idea is, get that thread working.
And, you know, maybe it's not all the way connected.
Maybe you still have somebody standing up with a floppy disk
and running to the other computer to make it all work,
but at least you have an idea.
And then you can broaden out that thread over time.
That's all.
SAM RANSBOTHAM: For our listeners,
floppy disks were things that people
had to put into computers to store information.
JACK BERKOWITZ: Yeah, thank you, Sam.
Thank you, Sam.
You can see it -- you'll actually see it on icons on old
Macs and old PCs, so, yeah.
SHERVIN KHODABANDEH: Jack, thank you so much.
This has been so wonderful and insightful.
This brings us to the next section of our show, where
we ask you five questions.
JACK BERKOWITZ: Great.
SHERVIN KHODABANDEH: And we expect some quick reactions
to these questions.
So I'll start.
What's your proudest AI moment?
JACK BERKOWITZ: My proudest AI moment
is when my algorithm went into production.
SHERVIN KHODABANDEH: Very good.
SAM RANSBOTHAM: That ties in well.
SHERVIN KHODABANDEH: What worries you about AI?
JACK BERKOWITZ: I think the next reoccurrence of the AI winter.
Having been through it the first time
and seeing that come short.
Let's not overpromise.
SHERVIN KHODABANDEH: Mm-hmm.
Your favorite activity that involves no technology.
JACK BERKOWITZ: Kayaking on the Chattahoochee River.
SAM RANSBOTHAM: Ah, been there, done that!
I've kayaked Chattahoochee many a time.
I'm actually from Atlanta, from Smyrna.
SHERVIN KHODABANDEH: The first career you wanted:
What did you want to be when you grew up?
JACK BERKOWITZ: I wanted to be an astronaut,
like every other kid born in the '60s.
SHERVIN KHODABANDEH: Your greatest wish
for AI in the future?
JACK BERKOWITZ: My greatest wish is that we
help people live better lives.
SHERVIN KHODABANDEH: Thank you.
Very insightful.
SAM RANSBOTHAM: Jack, great meeting you.
I think there's a lot here that people can learn,
particularly some of the details about how you're organized.
I think that's something a lot of people can learn from.
We really appreciate your taking the time.
JACK BERKOWITZ: Thanks, Sam.
I really appreciate the conversation.
SAM RANSBOTHAM: Thanks for joining us.
Next time, Shervin and I talk with Ameen Kazerouni,
chief data and analytics officer at Orangetheory Fitness.
ALLISON RYDER: Thanks for listening
to Me, Myself, and AI.
We believe, like you, that the conversation
about AI implementation doesn't start and stop
with this podcast.
That's why we've created a group on LinkedIn
specifically for listeners like you.
It's called AI for Leaders, and if you join us,
you can chat with show creators and hosts,
ask your own questions, share your insights,
and gain access to valuable resources about AI
implementation from MIT SMR and BCG.
You can access it by visiting mitsmr.com/AIforLeaders.
We'll put that link in the show notes,
and we hope to see you there.