AI Products Built on Structured Data: Text-to-SQL, Analytics, and What PMs Need to Know
TL;DR
While most AI PM conversations focus on RAG over documents and LLM chat interfaces, the largest untapped enterprise opportunity is structured data: the billions of rows sitting in Snowflake, Postgres, BigQuery, and Salesforce that nobody can query without writing SQL. Text-to-SQL products convert natural language questions into executable queries, and they are landing in every analytics, BI, and operations tool on the market. Building them well requires a different set of product decisions than unstructured AI apps, with accuracy as the core quality metric and data security as the enterprise gate. This guide covers the architecture, the product surface, and the roadmap decisions that determine whether a text-to-SQL product earns trust or gets abandoned.
The AI PM Minute
One tactic to make you a sharper AI PM, twice a week. 60 seconds to read. Free.
No fluff. Unsubscribe anytime.
Why Structured Data Is the Biggest AI Opportunity Most PMs Are Ignoring
The majority of AI product investment in 2025 and 2026 went into retrieval augmented generation over documents: PDFs, Slack messages, Notion pages, support tickets. The assumption was that enterprise value lived in unstructured text. It does not. It lives in databases.
A mid-sized enterprise typically stores customer data in a CRM, revenue data in a data warehouse, inventory in an ERP, HR records in a HRIS, and product telemetry in a time-series database. Each of these holds orders of magnitude more structured, queryable information than the company's document corpus. The problem is that accessing it requires SQL, a skill most business users do not have. The bottleneck is not the data: it is the query layer.
Text-to-SQL solves this directly. A user types "What was our average deal size in the northeast region last quarter, broken down by rep?" and the system converts that question into a SQL query, executes it against the relevant table, and returns a formatted answer. No data analyst required. No waiting for a report to be built.
This capability is now embedded in Tableau Pulse, ThoughtSpot AI, Microsoft Fabric Copilot, Databricks AI BI, and dozens of AI-native analytics startups. It is also being added to CRM copilots (Salesforce Einstein, HubSpot AI), ERP systems, and operations dashboards. The market is large and moving fast.
Business Intelligence Q&A
Revenue, pipeline, and performance questions answered without building a report. Replaces the BI team queue for ad hoc queries.
Financial Reporting
CFOs and FP&A teams querying actuals vs. budget, variance analysis, and cost breakdowns directly from the data warehouse.
Operational Analytics
Logistics, inventory, and fulfillment teams asking supply chain questions in plain language against their ERP data.
Customer 360 Queries
Sales and CS teams pulling customer history, usage patterns, and health signals from CRM and product telemetry without opening multiple dashboards.
HR and People Analytics
HR leaders querying headcount, attrition, and compensation data without waiting for a monthly report from the people analytics team.
Product Telemetry
Product managers asking 'how many users triggered this event in the last 7 days, segmented by plan tier' against the events table.
How Text-to-SQL Actually Works: The Technical Layer PMs Must Understand
You do not need to implement a text-to-SQL pipeline, but you do need to understand it at the level that shapes product decisions. The pipeline has five stages, and each stage introduces failure modes that determine your product quality.
1. Schema Ingestion
The system reads the database schema: table names, column names, data types, foreign key relationships, and sometimes sample rows. This is the 'context' the LLM will use to generate SQL. For large databases with hundreds of tables, loading the entire schema into the LLM context is not feasible.
2. Schema Selection
For large databases, a retrieval step identifies which tables are relevant to the user's question before the LLM call. This uses semantic search over the schema metadata. Poor schema selection is the most common cause of wrong queries on large enterprise databases.
3. SQL Generation
The LLM receives the user question, the relevant schema, any business rules, and generates a SQL query. This is where most products compete on prompt engineering, few-shot examples, and model selection.
4. Execution and Validation
The generated SQL is executed against the database. Many products add a validation step: check that the SQL is syntactically valid before running it, and add a row limit to prevent runaway queries. If execution fails, some products retry with error context.
5. Result Formatting
Raw query results are formatted into a readable answer: a table, a number, a chart, or a natural language summary. The choice of output format is a product decision, not a technical one.
Two architecture patterns dominate. The first is in-context schema injection: load the entire schema into the system prompt. This works well for small schemas (under 50 tables) and is simpler to build and test. The second is schema retrieval and selection: use semantic search to find the most relevant tables for a given question, then inject only those tables into the LLM context. This scales to hundreds or thousands of tables but requires a well-maintained schema metadata layer.
The four failure modes that kill user trust in text-to-SQL products:
Hallucinated column names
The LLM invents column names that do not exist. Mitigation: strict schema grounding, post-generation SQL parsing to verify column names against the actual schema before execution.
Wrong aggregations
SUM used instead of AVG, or COUNT(DISTINCT) omitted when deduplication is needed. Mitigation: business glossary annotations on the schema defining what each metric means.
Missing joins
The LLM queries one table when the question requires joining two. Mitigation: explicit foreign key annotations and example queries in the schema context.
Data freshness confusion
Users assume data is current when the warehouse is 24 hours stale. Mitigation: surface data freshness timestamps in every query result.
The Product Surface: What Users Actually Need
A text-to-SQL product looks simple: a text box, a result. But the product decisions underneath that interface determine whether the tool earns daily use or gets written off after the first wrong answer.
The first decision is which user you are building for. Data analysts and business users want fundamentally different things, and conflating them is one of the most common early-stage mistakes in analytics AI products.
Data Analyst Persona
- Primary need: Speed up query writing, not replace it
- Key feature: Show and edit the SQL before running it
- Trust signal: I can see the query and verify it is correct
- Red flag: Black box answers with no SQL visibility
- Secondary value: Auto-complete, schema exploration, faster joins
Business User Persona
- Primary need: Get an answer without knowing SQL exists
- Key feature: Plain language answer with chart visualization
- Trust signal: The answer looks right and matches what I already know
- Red flag: Showing SQL scares them and implies they should understand it
- Secondary value: Saved queries, scheduled reports, natural follow-up questions
Beyond the persona split, the core product features that determine retention are:
Query history and sharing
Teams ask similar questions. Saving and sharing successful queries turns individual productivity into team productivity. It also creates a golden set for evaluations.
Confidence signals
When the system is uncertain (ambiguous question, sparse schema, multi-hop join required), surfacing that uncertainty prevents users from treating a likely-wrong answer as fact.
Explanation mode
'This query sums revenue from the orders table, filtered to Q3, grouped by sales rep' builds trust incrementally. Business users gain intuition about what the product can and cannot do.
Scheduled reports
Converting a successful ad hoc query into a recurring report delivered by email or Slack is a low-effort, high-retention feature. It moves the product from reactive to proactive.
Data Security and Access Control: The Enterprise Blocker
Security review kills more analytics AI deals than any product quality issue. Enterprise procurement teams ask one core question before anything else: can a user ask your AI to return data they are not supposed to see?
This is not a hypothetical concern. A text-to-SQL product that ignores row-level security will generate a SQL query without the WHERE clause restricting data to the user's access scope. A sales rep asking "show me all accounts over $500K ARR" should only see accounts in their territory, not the entire company's customer base. If the LLM does not know about these access rules, it will not apply them.
The four security requirements enterprise security teams will evaluate before signing:
Row-Level Security Enforcement
The product must inject the user's access context (user ID, role, region, tenant ID) into every generated query. This requires maintaining a permissions layer that is synchronized with the source database's access controls. Stale permissions are a compliance gap.
Column Masking and Sensitive Data Controls
Certain columns must never appear in results regardless of what users ask: SSNs, credit card numbers, salary data, health records. The product needs an explicit column allowlist or blocklist per schema, enforced before query execution.
Data Egress Controls
Prevent users from exfiltrating large datasets through natural language. 'Export all customer emails and phone numbers' should be blocked or require additional authorization. Row limits, download restrictions, and audit logging are the standard controls.
Audit Logging
Every query, who ran it, what SQL was generated, and what data was returned must be logged for compliance and incident response. Enterprise security teams will ask for this in the security questionnaire. Build it before the first enterprise conversation, not after.
On-premise or VPC deployment is frequently required for data warehouses containing regulated data (financial, health, PII-heavy). A SaaS deployment that sends schema context and query results through your infrastructure is a dealbreaker for many enterprise security teams. Plan your deployment architecture to support private cloud before the first enterprise pilot, not after the first security review rejection.
Go Deeper in the AI PM Masterclass
The masterclass covers how to evaluate technical architectures, spot product risks early, and make build vs. buy decisions on AI infrastructure, taught live by a Salesforce Sr. Director PM.
Building Your Text-to-SQL Product: The PM Roadmap
The right beachhead for a text-to-SQL product is a single team with a single schema. Not the entire company data warehouse, not every department's data: one schema, one use case, one set of users. Going cross-domain too early multiplies schema complexity, increases failure rates on ambiguous cross-schema questions, and makes it impossible to establish a meaningful accuracy baseline.
Accuracy is the product. This is the defining difference between text-to-SQL and most other AI product categories. A content generation tool where the output is "pretty good" still delivers value. A SQL generation tool that returns a number that is 10% wrong gives a sales leader confidence in a revenue forecast that is incorrect. The trust loss from a single wrong answer can take weeks to recover. Build with this in mind from day one.
The five-stage accuracy improvement loop that strong text-to-SQL products run continuously:
1. Build a golden evaluation set
Before launching, create 50 to 100 natural language questions and their correct SQL answers for your target schema. This is tedious and cannot be automated, but it is the foundation of everything that follows.
2. Establish a baseline accuracy score
Run the golden set against your current system. Measure execution success rate, semantic accuracy (did the SQL answer the right question), and result correctness. Set the baseline before shipping.
3. Instrument production queries
Log all production queries, generated SQL, execution outcomes, and user corrections. User corrections are your most valuable signal: they show exactly where the system fails and what the correct behavior is.
4. Add failed queries to the evaluation set
When a production query fails or gets corrected, add the question and correct SQL to the golden set. The evaluation set grows continuously from real usage patterns.
5. Iterate on schema annotations and few-shot examples
Most accuracy improvements come not from changing the LLM but from improving the schema context: better column descriptions, explicit business rules, example queries for common patterns. Treat the schema layer as a product asset.
Common mistakes in the first six months: building for managers before frontline users are hooked, launching without a clear schema ownership model (who updates the annotations when the database changes?), and underestimating how much business logic is implicit in the data model. Every company has column names like "rev_adj_net_q" that mean something specific internally and nothing to an LLM without a glossary entry.
What Text-to-SQL Products Demand From AI PMs
Building and shipping a text-to-SQL product successfully requires a specific combination of AI PM skills that are different from RAG or agent products. You need enough SQL fluency to understand what a generated query is doing and whether it is correct. You need a mental model for database security and access control. And you need to run a disciplined evaluation practice, not just ship and monitor anecdotally.
The PM who succeeds here is one who can sit with a data engineer, read a schema, understand a query plan, and translate that into a product requirement. They do not need to write production SQL daily, but they need to read it fluently enough to catch quality issues before users do.
SQL literacy
Read and understand generated queries. Identify when a query is semantically correct vs. syntactically valid. Spot missing GROUP BY clauses, wrong aggregation functions, and overly broad JOINs.
Schema and data modeling intuition
Understand primary keys, foreign keys, normalization levels, and why denormalized warehouse schemas behave differently from OLTP schemas. This determines where schema annotation effort is highest value.
Evaluation design
Build and maintain golden evaluation sets. Define what 'correct' means for ambiguous questions. Set accuracy targets and track them over time, not just launch and move on.
Security requirements literacy
Understand row-level security, column masking, and audit logging well enough to spec them into the product and answer questions in an enterprise security review without deflecting to engineering.
The AI PM Masterclass covers each of these skill areas through hands-on exercises with real data schemas, eval design sessions, and product strategy for technical AI products. If you are building in the analytics AI space and want the technical fluency to do it credibly, the December cohort is the fastest path.
Get the Technical Fluency to Ship AI Products
The AI PM Masterclass teaches the technical depth, evaluation frameworks, and product strategy that structured data AI products demand. Taught live by a Salesforce Sr. Director PM who has built and shipped AI features at scale.
Related Articles
Before you go: get the AI PM Minute
One tactic to make you a sharper AI PM, twice a week. 60 seconds to read. Free.
No fluff. Unsubscribe anytime.