You might be thinking, “I thought AI LLMs sucked at basic math and generated hands with 6 fingers. How do you want them to do visualizations that are the basis for really high stakes product decisions?” That’s true, but LLMs are amazing at communicating with the world through code. What that means for data analysis is LLMs can take in large amounts of data or interact with large amounts of raw input data with languages like Python and SQL - languages that the LLM can even run. Claude and other data analysis tools can actually run Python on CSVs. I’ve seen demos where an LLM can run SQL in another database. So the code is being run, and the data exists in those environments where it’s very reliable and precise. The LLM is communicating and analyzing using that code. On the other end, to visualize, LLMs can write code in the form of HTML and CSS, which also allows you to quickly iterate and say, “Hey, I don’t like this, I do like that.” Both in and out, LLMs can communicate through code which is perfect for this use case. Hilary Gridley brings up a crucial point about combining quantitative and qualitative data. She’s always been a big fan of this approach, and LLMs can do a really good job with that in a few ways: First, they can turn actual qualitative data into quantitative data. You can take things that are qualitative in nature - app store reviews, reddit subreddits, customer service tickets - and turn them into quantitative data that you can manipulate and get really interesting insights from at scale. Second, they’re really helpful at contextualizing quantitative data. If you spend a lot of time with numbers, saying, “That’s interesting, I’m seeing things move, but I’m trying to understand why,” LLMs can help you do analysis around that as well. Third, they’re very good at sense-checking whether a quantitative focus is giving you the full picture. We all know we shouldn’t be assessing the results of an A/B test in a vacuum, yet often we do. We know that if we put a ton of people in the top of the funnel, we’re going to start seeing problems down in the funnel if those aren’t well-qualified people. LLMs can help you contextualize that analysis to understand if you’re getting the full picture. ➡️ LLMs aren’t doing the math directly - they’re writing and running code that does precise calculations. This solves the reliability problem while giving you the natural language interface and rapid iteration capabilities that make complex analysis accessible to PMs. Check out Hilary’s course (not sponsored, she’s just awesome).