I’m Randi Griffin, I’m a Lead Data Scientist in the Boston office with BCG X.
Data scientists are constantly having to learn new tools and techniques
but there are fundamentals that apply across every problem.
Take GenAI as an example.
Data scientists everywhere are having to ramp up on this new technology.
And there is a lot about it that is new,
but the fundamentals still apply.
To develop applications at scale,
you need high-quality data, robust infrastructure,
evaluation and monitoring of system performance.
One of the biggest challenges around GenAI
is bridging the gap between proofs of concept and scaled solutions.
A major driver of this is gaps in tooling and capabilities around testing
and evaluating GenAI systems
to ensure they are consistently delivering the intended value.
To help solve this, we’ve been developing some tools and playbooks
focused on testing and evaluation of GenAI systems.
I’ll tell you about one that I’m particularly excited about.
We’ve developed a software library for using GenAI
to automate the process of testing and evaluating another GenAI system.
When I say automate,
I don’t mean you just push a button and it’s one and done.
There’s still a lot of thought and expertise that has to go into developing targeted,
effective test and evals for a specific GenAI use case.
But what our tool does is
it solves a lot of the engineering challenges, which allows teams
to focus on developing a tailored solution for their business context.