This interview took place on December 4th 2024 over a video conference. The interview was conducted as a part of the Lead by Numbers project, a book for leaders who want to develop a culture of data and begin their journey towards AI.
Thanks for joining the interview today, I’m excited to learn about your career and the data organizations you’ve built. To start, I’d like to ask a personal question; you studied mathematics in college, what interested you most about math and what inspired you to pursue it as a major?
It was a lot of different things. I had a phenomenal AP calculus teacher. I’ve also always loved math and logic. I was the child that when we would go on road trips, I’d ask my parents to give me long division problems to do in the back of the car. So, it’s been a lifelong passion. With math, I think that there’s a lot of beauty. I’ve read many books on different aspects of math. Everything from the actual mathematics to the history of math, like the history of zero, which is actually quite interesting. As well as about mathematicians themselves.
I’ve also always been passionate about computer science. I took AP Computer Science in high school, and went on to study Computer Engineering at the University of Michigan. Midway through college, when I moved back to Colorado, it made more sense for me to change my major from Computer Engineering to math. So it was kind of happenstance that I actually got my degree in math.
I did my senior thesis on topology and how it relates to quantum computing. Since I was interested in quantum computing, I pursued work in the government contracting world. I was interested in how to ensure that we could protect against the breaking of our entire encryption system. So that was what led me to Northrop Grumman initially.
However, I never ended up doing anything with Quantum Computing. I right off the bat ended up working with an individual who had a PhD in machine learning. He became one of my greatest mentors and is still a great friend. He’s since retired, but I ended up working with him for a long time. And that’s what drove me into the data and analytics realm and doing AI data processing, among other cutting edge technology work early in my career.
You’re involved with all things, data governance, management, architecture, and literacy. Which of these is most valuable and which is the most difficult to optimize?
I think it’s hard to say which one is most important. I think they’re all incredibly important, which is why we’re investing in all of them. I mean this in the sense that you need to have data literacy so that you’re improving skills among both data professionals and non-data professionals. Data literacy is important so that they understand the realm of possibility for how to leverage data and what the different practices are within the field of data.
For data management, there is data governance, metadata management, and core data management. All of these areas are incredibly important for structuring data and ensuring that you’re making it usable for the business.
And then there is the data utilization piece, such as data analytics, AI, data visualization. These are what most people see at the end of the process. And so a lot of people view them as most valuable, but you can’t actually do any of those without investment in data management, data architecture, and data ecosystem.
The most difficult of these to achieve for a company is probably data governance. This is because it requires rigor and a methodical nature. I always liken it to; ‘you can clean your house once, but that doesn’t mean it’s going to stay clean.’ The same can be said with data; you can clean it once, but it’s not going to stay clean unless you have processes and practices in place to ensure that you’re maintaining data quality. That’s why data governance is so important, but it really takes buy-in from the entire company and the entire company placing value on the need for data governance.
“You can clean it once, but it’s not going to stay clean unless you have processes and practices in place to ensure that you’re maintaining data quality.”
Data governance has been getting a lot more focus though because of AI. Just recently, the Microsoft CEO at one of their investor calls was talking about how important data governance is and how much they’re investing in data governance across the board. So I think that the industry is really starting to appreciate the value that data governance brings, which is improving the difficult nature of data governance.
Would it be possible to discuss the most memorable data initiative you oversaw?
A great example was our response to COVID at Northrop Grumman. When COVID occurred, we were set up as a company to immediately respond through data and analytics. We needed to understand the impacts of COVID on our supply chain. And since we already had a data and analytics organization established, we were able to quickly develop insights and identify supply chain impacts, such as which suppliers were open and which were closed.
Another critical area regarding COVID, was understanding the relevant laws and regulations. At the time, the situation was developing quickly, but our team was able to rapidly respond and develop an entire system that allowed HR to manage the safety of our employees and adhere to the relevant laws and regulations.
We also leveraged AI and data science models in order to forecast and better predict the impacts to our supply chain. We created AI models of COVID progression to understand the greatest contributors to the propagation of COVID. From this we were able to anticipate the spread of COVID, and as a result, reduce its impacts.
There was a lot that we did in a very short time frame in order to establish data collection processes, manage data, and publish dashboards. We standardized how we were looking at all of our information, in order to develop data science models and mitigate the impacts of COVID.
To switch gears to F5, would it be possible to discuss your vision for data analytics?
I think that it’s no different than probably any other company. We can talk about it from a general perspective and why we’re striving to mature the utilization of data in order to drive business results and competitive differentiation for the company. So that means investment across data enablement functions. Data enablement functions include data management, coordination, metadata, data governance, architecture, infrastructure and utilization. Data utilization includes data science, AI data visualization, data analysis, data literacy and change management.
What has been your process for leveraging cutting edge technologies like AI and machine learning. How do you assess AI use cases? How do you ensure success while implementing these initiatives?
For this, I think it’s so important to develop strong relationships across the business. It is also important to strive to understand every facet of the company which allows for the identification of opportunities to utilize data as a solution. So that really means meeting with all of the leaders across the company as much as possible and to understanding where they are struggling, and how data can improve that.
We base our prioritization on an assessment of strategic value creation, our business partner support and also data accessibility. Data accessibility is especially important because while there might be a great concept of how to leverage cutting edge technologies, we can’t do anything about it if the data doesn’t exist. However, that doesn’t necessarily mean the initiative can’t be done at some point in the future. Additionally, some planning and investment in our data ecosystem may allow the target use case in the future.
What are the greatest barriers to successfully realizing value from new cutting edge technologies?
I think the greatest challenge right now, and I’ve been in the AI space for 20 plus years, is in how we process data so that we’re able to do advanced use cases, like talk-to-text. Many of our overarching data infrastructure investments have enabled advanced AI use cases that we’re able to implement now.
With the release of ChatGPT, I think that the biggest challenge has been the over-hype of AI and individuals asking for AI without really understanding if that is the solution that they need. They’re asking for a solution as opposed to talking about a problem and what they actually need. And it may be that AI could be the answer, but could also be that they might actually need something completely different. But they’re asking for AI because it’s so hyped right now. I think it’s important for everyone, especially professionals who are going into the field, to understand how to leverage AI, its best use cases, and when not to leverage it or when leveraging it will not be successful.
Setting expectations, and having scoping discussions early in the process is also important in understanding the underlying business need, the actual requirements and the appropriate solution. This is opposed to saying, ‘hey, we need a hammer’, when you might actually need a screwdriver.
Is AI the key emerging technology you’re focused on? Are there any other emerging technologies that you’re looking at?
I think AI is definitely the top emerging technology that we’re focusing on right now.
Where do you see the most value in your data initiatives? Is it in AI or is it in more fundamental capabilities, like report automation?
I see so much value in all of it. AI definitely has significant ROI. And when you’re applying it to the right use case, it’s phenomenal. With AI, we can do significant risk buy-down, cost reduction and revenue maximization. There’s just so much that can be done with it. It can also improve people’s jobs, how they work and create an overall happier workplace. AI can also be applied within other fields where there are huge benefits to humans in general. So there’s tremendous potential value in AI when applied to the right use cases.
However, there’s also value, sometimes overlooked, in things like visualization and dashboarding, data management, and data architecture. All of these more foundational pieces are important and are required in order to enable AI use cases. Sometimes the value of the products that aren’t at the end of the data process don’t get as much acknowledgement as end products that everyone sees, like visualizations.
What types of use cases is your team implementing with AI and machine learning?
Specifically within generative AI, we use a lot of RAG (Retrieval-Augmented Generation) models. We’ve been doing a lot with prompt engineering as well. I think that’s an important focus area. There’s also a lot to be said about just traditional, it’s funny to call them traditional now, neural networks and statistical analytics. There are also other realms that fall under AI and machine learning. There are some great use cases for all of those still as well. Also worth mentioning are recommendation systems, which are still really important to us.
When you’re building a new team, what characteristics do you look for when recruiting new team members?
One of the first things I look for is innate curiosity. The other is communication, because being successful in data and analytics requires an ability to communicate with business partners to define their challenges and then solution against those challenges. However, effectively understanding the problem takes curiosity and communication.
When a stakeholder says, ‘I want the system to be green’, we shouldn’t just think, ‘oh, they want it painted green’. We should rather ask questions about why it’s important that it be green and what that means to them. Because maybe they don’t actually care what color it is, but rather what they need is energy efficiency.
Our words have multiple meanings, so you really have to understand the root of the challenges that people face. The power of data and analytics is understanding the root of those challenges and then being able to solution against them.
“Our words have multiple meanings, so you really have to understand the root of the challenges that people face. The power of data and analytics is understanding the root of those challenges and then being able to solution against them.”
What is your process for developing a successful team and data culture?
I have four primary values that I’ve always impressed upon all of my teams. These are strategic value creation, top-notch quality, inclusive teamwork, and then systems thinking or consultative skills—I usually call it consultative skills. The consultative skills are the curiosity, and the communication skills are what’s needed to get to the premise of what somebody is asking for. That’s the ability to actually understand the value of the solution. and then be able to communicate that value.
I recently had a conversation with an individual about strategic value. I was asking them about the value of the project they were proposing. They were talking about technical value creation and were saying how they’ve used a new skill and, as a result, improved a certain model by X%. It was great, but it wasn’t clear what value they were demonstrating to the business. It wasn’t clear whether they were enhancing revenue, reducing risk, or conveying any purpose other than demonstrating the new skillset. And so, it’s important for the team to be able to demonstrate the business value in their data products.
Demonstrating top-notch quality is important because if we release any data product that isn’t reliable, where the data has not been verified, then we lose trust from our stakeholders, our business partners, and the entire organization. The whole reason they’re coming to us is to have data insights and gain more information. So we have to ensure quality in everything that we produce or else our products won’t be valuable to the company.
Inclusive teamwork is also important because a team can have one superstar, but if you’re not working as a team, then it’s not just productivity that suffers, it’s also team engagement and overall team happiness. So I think inclusivity is important when building a team. The ideal situation is one in which individuals ask each other questions, solution as a team and ultimately enjoy spending time together.
Regarding the challenges of building a cohesive team, I think it becomes a lot easier to reduce any type of challenges by establishing core values that you impress and reiterate time and time again. This way the team understands your expectations, which in the long term, enables them to operate together.
What was your process for initially starting a new data program at Northrop Grumman? And what were the greatest challenges while initiating that program?
Initiating the data practice at Northrop Grumman was natural, in the sense that I had already worked my way up the organization. I had started in AI and data R&D within the intelligence community, and then moved between various positions and sectors of the business. This was a part of an intentional effort to broaden my exposure to all of our different customers and program types. Northrop operates significantly differently at every site. So it was important to gain more of an understanding of each site.
When I went into management, I was within one of our four business sectors and had a portfolio of data and analytics related projects for our customers. After a briefing one day, the president of one of the business sectors asked that I start a similar data transformation for his organization. So I added that to our portfolio and we started managing analytics projects for that business sector. I had a phenomenal team, and as a result, the president of another business sector asked that we repeat the process for his organization.
It was then that corporate decided they needed a dedicated Data and Analytics organization. And so they reached out to me and asked if I’d create that organization. Because of that progression, I had already set a vision for what needed to occur. I already knew what the pain points were across the company and where resources needed to be invested. So before even accepting the position, I put together a vision and discussed it with who would become my boss and said, this is what I’m envisioning. After stepping into that role, it was all about execution.
After starting the Data and Analytics organization, we hired as fast as possible to get all of the individuals into the right roles so that we could begin executing. One leg of the organization was focused on building a data fabric, and establishing data governance. While the other was focused on the enablement of data through data science, data visualization, etc.
In addition to foundational data management, analytics and visualization, we also had a focus on ethics as well. We appointed a Chief Responsibility Officer, who was focused on data ethics. This was great because once we wanted to get into AI, we were ready to go since we already had someone focused on AI and data ethics.
Regarding execution, I think the biggest challenge in building any new organization is the setting of expectations among the business at large. This is because if a function has never existed before and now it exists, everyone has different perspectives for what they’re expecting from it. You have to establish processes for how you expect other functions to also engage with you. This constant setting of expectations is critical.
How can an organization go from zero to one when establishing data competency? What would you tell them and where should they start?
Step one is establishing a data and analytics organization. This is followed by getting the right leader in that role who has a vision that aligns with what the company’s needs. The reason for that is because then you have one person on point who is responsible for the execution of building the data function. They are then responsible for creating and maturing the practice for the company. So I think you need to have one responsible entity for the company to be successful. Otherwise, the company will just take it on by asking every function to improve, without any real accountability. When this happens, it’s not one person who is actually accountable for the outcomes. And as a result you see slower and more ineffective results by just making it the responsibility of the entire company.
What did you learn from other executives when building a new data practice? What were the greatest challenges to building a successful data and analytics practice and culture?
What I learned from the other executives was how to identify opportunities in order to leverage data and analytics to accelerate the company. What I also learned was how passionate so many people are about data and analytics. This is great because that passion makes all those individuals advocates for data and analytics in the company. I never want to be the only advocate for data and analytics in a company.
And so, as much as possible, I think it’s important to bring on other advocates as well. This includes other executives who are interested in data and analytics or who I can partner with to define the roadmap. If the solution benefits an executive’s function, then it’s always best to have them act as a change champion. It is great to have individuals who will benefit from telling their story, broadcasting it and becoming an advocate for the value of data and analytics. I think that becomes tremendously powerful.
“It is great to have individuals who will benefit from telling their story, broadcasting it and becoming an advocate for the value of data and analytics.”