TECHNICAL DEEP DIVE

AI Products Built on Structured Data: Text-to-SQL, Analytics, and What PMs Need to Know

By Institute of AI PM·14 min read·Nov 17, 2026

TL;DR

While most AI PM conversations focus on RAG over documents and LLM chat interfaces, the largest untapped enterprise opportunity is structured data: the billions of rows sitting in Snowflake, Postgres, BigQuery, and Salesforce that nobody can query without writing SQL. Text-to-SQL products convert natural language questions into executable queries, and they are landing in every analytics, BI, and operations tool on the market. Building them well requires a different set of product decisions than unstructured AI apps, with accuracy as the core quality metric and data security as the enterprise gate. This guide covers the architecture, the product surface, and the roadmap decisions that determine whether a text-to-SQL product earns trust or gets abandoned.

The AI PM Minute

One tactic to make you a sharper AI PM, twice a week. 60 seconds to read. Free.

No fluff. Unsubscribe anytime.

Why Structured Data Is the Biggest AI Opportunity Most PMs Are Ignoring

The majority of AI product investment in 2025 and 2026 went into retrieval augmented generation over documents: PDFs, Slack messages, Notion pages, support tickets. The assumption was that enterprise value lived in unstructured text. It does not. It lives in databases.

A mid-sized enterprise typically stores customer data in a CRM, revenue data in a data warehouse, inventory in an ERP, HR records in a HRIS, and product telemetry in a time-series database. Each of these holds orders of magnitude more structured, queryable information than the company's document corpus. The problem is that accessing it requires SQL, a skill most business users do not have. The bottleneck is not the data: it is the query layer.

Text-to-SQL solves this directly. A user types "What was our average deal size in the northeast region last quarter, broken down by rep?" and the system converts that question into a SQL query, executes it against the relevant table, and returns a formatted answer. No data analyst required. No waiting for a report to be built.

This capability is now embedded in Tableau Pulse, ThoughtSpot AI, Microsoft Fabric Copilot, Databricks AI BI, and dozens of AI-native analytics startups. It is also being added to CRM copilots (Salesforce Einstein, HubSpot AI), ERP systems, and operations dashboards. The market is large and moving fast.

Business Intelligence Q&A

Revenue, pipeline, and performance questions answered without building a report. Replaces the BI team queue for ad hoc queries.

Financial Reporting

CFOs and FP&A teams querying actuals vs. budget, variance analysis, and cost breakdowns directly from the data warehouse.

Operational Analytics

Logistics, inventory, and fulfillment teams asking supply chain questions in plain language against their ERP data.

Customer 360 Queries

Sales and CS teams pulling customer history, usage patterns, and health signals from CRM and product telemetry without opening multiple dashboards.

HR and People Analytics

HR leaders querying headcount, attrition, and compensation data without waiting for a monthly report from the people analytics team.

Product Telemetry

Product managers asking 'how many users triggered this event in the last 7 days, segmented by plan tier' against the events table.

How Text-to-SQL Actually Works: The Technical Layer PMs Must Understand

You do not need to implement a text-to-SQL pipeline, but you do need to understand it at the level that shapes product decisions. The pipeline has five stages, and each stage introduces failure modes that determine your product quality.

1. Schema Ingestion

The system reads the database schema: table names, column names, data types, foreign key relationships, and sometimes sample rows. This is the 'context' the LLM will use to generate SQL. For large databases with hundreds of tables, loading the entire schema into the LLM context is not feasible.

2. Schema Selection

For large databases, a retrieval step identifies which tables are relevant to the user's question before the LLM call. This uses semantic search over the schema metadata. Poor schema selection is the most common cause of wrong queries on large enterprise databases.

3. SQL Generation

The LLM receives the user question, the relevant schema, any business rules, and generates a SQL query. This is where most products compete on prompt engineering, few-shot examples, and model selection.

4. Execution and Validation

The generated SQL is executed against the database. Many products add a validation step: check that the SQL is syntactically valid before running it, and add a row limit to prevent runaway queries. If execution fails, some products retry with error context.

5. Result Formatting

Raw query results are formatted into a readable answer: a table, a number, a chart, or a natural language summary. The choice of output format is a product decision, not a technical one.

Two architecture patterns dominate. The first is in-context schema injection: load the entire schema into the system prompt. This works well for small schemas (under 50 tables) and is simpler to build and test. The second is schema retrieval and selection: use semantic search to find the most relevant tables for a given question, then inject only those tables into the LLM context. This scales to hundreds or thousands of tables but requires a well-maintained schema metadata layer.

The four failure modes that kill user trust in text-to-SQL products:

Hallucinated column names

The LLM invents column names that do not exist. Mitigation: strict schema grounding, post-generation SQL parsing to verify column names against the actual schema before execution.

Wrong aggregations

SUM used instead of AVG, or COUNT(DISTINCT) omitted when deduplication is needed. Mitigation: business glossary annotations on the schema defining what each metric means.

Missing joins

The LLM queries one table when the question requires joining two. Mitigation: explicit foreign key annotations and example queries in the schema context.

Data freshness confusion

Users assume data is current when the warehouse is 24 hours stale. Mitigation: surface data freshness timestamps in every query result.

The Product Surface: What Users Actually Need

A text-to-SQL product looks simple: a text box, a result. But the product decisions underneath that interface determine whether the tool earns daily use or gets written off after the first wrong answer.

The first decision is which user you are building for. Data analysts and business users want fundamentally different things, and conflating them is one of the most common early-stage mistakes in analytics AI products.

Data Analyst Persona

  • Primary need: Speed up query writing, not replace it
  • Key feature: Show and edit the SQL before running it
  • Trust signal: I can see the query and verify it is correct
  • Red flag: Black box answers with no SQL visibility
  • Secondary value: Auto-complete, schema exploration, faster joins

Business User Persona

  • Primary need: Get an answer without knowing SQL exists
  • Key feature: Plain language answer with chart visualization
  • Trust signal: The answer looks right and matches what I already know
  • Red flag: Showing SQL scares them and implies they should understand it
  • Secondary value: Saved queries, scheduled reports, natural follow-up questions

Beyond the persona split, the core product features that determine retention are:

Query history and sharing

Teams ask similar questions. Saving and sharing successful queries turns individual productivity into team productivity. It also creates a golden set for evaluations.

Confidence signals

When the system is uncertain (ambiguous question, sparse schema, multi-hop join required), surfacing that uncertainty prevents users from treating a likely-wrong answer as fact.

Explanation mode

'This query sums revenue from the orders table, filtered to Q3, grouped by sales rep' builds trust incrementally. Business users gain intuition about what the product can and cannot do.

Scheduled reports

Converting a successful ad hoc query into a recurring report delivered by email or Slack is a low-effort, high-retention feature. It moves the product from reactive to proactive.

Data Security and Access Control: The Enterprise Blocker

Security review kills more analytics AI deals than any product quality issue. Enterprise procurement teams ask one core question before anything else: can a user ask your AI to return data they are not supposed to see?

This is not a hypothetical concern. A text-to-SQL product that ignores row-level security will generate a SQL query without the WHERE clause restricting data to the user's access scope. A sales rep asking "show me all accounts over $500K ARR" should only see accounts in their territory, not the entire company's customer base. If the LLM does not know about these access rules, it will not apply them.

The four security requirements enterprise security teams will evaluate before signing:

Row-Level Security Enforcement

The product must inject the user's access context (user ID, role, region, tenant ID) into every generated query. This requires maintaining a permissions layer that is synchronized with the source database's access controls. Stale permissions are a compliance gap.

Column Masking and Sensitive Data Controls

Certain columns must never appear in results regardless of what users ask: SSNs, credit card numbers, salary data, health records. The product needs an explicit column allowlist or blocklist per schema, enforced before query execution.

Data Egress Controls

Prevent users from exfiltrating large datasets through natural language. 'Export all customer emails and phone numbers' should be blocked or require additional authorization. Row limits, download restrictions, and audit logging are the standard controls.

Audit Logging

Every query, who ran it, what SQL was generated, and what data was returned must be logged for compliance and incident response. Enterprise security teams will ask for this in the security questionnaire. Build it before the first enterprise conversation, not after.

On-premise or VPC deployment is frequently required for data warehouses containing regulated data (financial, health, PII-heavy). A SaaS deployment that sends schema context and query results through your infrastructure is a dealbreaker for many enterprise security teams. Plan your deployment architecture to support private cloud before the first enterprise pilot, not after the first security review rejection.

Go Deeper in the AI PM Masterclass

The masterclass covers how to evaluate technical architectures, spot product risks early, and make build vs. buy decisions on AI infrastructure, taught live by a Salesforce Sr. Director PM.

Building Your Text-to-SQL Product: The PM Roadmap

The right beachhead for a text-to-SQL product is a single team with a single schema. Not the entire company data warehouse, not every department's data: one schema, one use case, one set of users. Going cross-domain too early multiplies schema complexity, increases failure rates on ambiguous cross-schema questions, and makes it impossible to establish a meaningful accuracy baseline.

Accuracy is the product. This is the defining difference between text-to-SQL and most other AI product categories. A content generation tool where the output is "pretty good" still delivers value. A SQL generation tool that returns a number that is 10% wrong gives a sales leader confidence in a revenue forecast that is incorrect. The trust loss from a single wrong answer can take weeks to recover. Build with this in mind from day one.

The five-stage accuracy improvement loop that strong text-to-SQL products run continuously:

1. Build a golden evaluation set

Before launching, create 50 to 100 natural language questions and their correct SQL answers for your target schema. This is tedious and cannot be automated, but it is the foundation of everything that follows.

2. Establish a baseline accuracy score

Run the golden set against your current system. Measure execution success rate, semantic accuracy (did the SQL answer the right question), and result correctness. Set the baseline before shipping.

3. Instrument production queries

Log all production queries, generated SQL, execution outcomes, and user corrections. User corrections are your most valuable signal: they show exactly where the system fails and what the correct behavior is.

4. Add failed queries to the evaluation set

When a production query fails or gets corrected, add the question and correct SQL to the golden set. The evaluation set grows continuously from real usage patterns.

5. Iterate on schema annotations and few-shot examples

Most accuracy improvements come not from changing the LLM but from improving the schema context: better column descriptions, explicit business rules, example queries for common patterns. Treat the schema layer as a product asset.

Common mistakes in the first six months: building for managers before frontline users are hooked, launching without a clear schema ownership model (who updates the annotations when the database changes?), and underestimating how much business logic is implicit in the data model. Every company has column names like "rev_adj_net_q" that mean something specific internally and nothing to an LLM without a glossary entry.

What Text-to-SQL Products Demand From AI PMs

Building and shipping a text-to-SQL product successfully requires a specific combination of AI PM skills that are different from RAG or agent products. You need enough SQL fluency to understand what a generated query is doing and whether it is correct. You need a mental model for database security and access control. And you need to run a disciplined evaluation practice, not just ship and monitor anecdotally.

The PM who succeeds here is one who can sit with a data engineer, read a schema, understand a query plan, and translate that into a product requirement. They do not need to write production SQL daily, but they need to read it fluently enough to catch quality issues before users do.

SQL literacy

Read and understand generated queries. Identify when a query is semantically correct vs. syntactically valid. Spot missing GROUP BY clauses, wrong aggregation functions, and overly broad JOINs.

Schema and data modeling intuition

Understand primary keys, foreign keys, normalization levels, and why denormalized warehouse schemas behave differently from OLTP schemas. This determines where schema annotation effort is highest value.

Evaluation design

Build and maintain golden evaluation sets. Define what 'correct' means for ambiguous questions. Set accuracy targets and track them over time, not just launch and move on.

Security requirements literacy

Understand row-level security, column masking, and audit logging well enough to spec them into the product and answer questions in an enterprise security review without deflecting to engineering.

The AI PM Masterclass covers each of these skill areas through hands-on exercises with real data schemas, eval design sessions, and product strategy for technical AI products. If you are building in the analytics AI space and want the technical fluency to do it credibly, the December cohort is the fastest path.

Get the Technical Fluency to Ship AI Products

The AI PM Masterclass teaches the technical depth, evaluation frameworks, and product strategy that structured data AI products demand. Taught live by a Salesforce Sr. Director PM who has built and shipped AI features at scale.

Before you go: get the AI PM Minute

One tactic to make you a sharper AI PM, twice a week. 60 seconds to read. Free.

No fluff. Unsubscribe anytime.