From AI research chat to an inspectable decision loop

I kept asking AI to research B2B SaaS companies. The answers were usually good.

A company would have a confusing packaging model. Another would have an expansion opportunity hidden inside its customer base. Another seemed to be adding AI features without a clear connection to activation, retention, or revenue.

The model could find patterns quickly. It could summarize a product, interpret pricing, inspect public signals, and suggest where a conversation might be useful.

But something felt wrong. Every company looked interesting. Every answer sounded plausible.

And after a few days, I could no longer tell which conclusions came from strong evidence, which ones came from weak signals, and which ones simply sounded convincing because the model wrote them well.

The problem was not that AI could not research. The problem was that the research had no operating system around it.

Chat was useful, but it was not a workflow

A chat session is a good place to think. It is a poor place to run a repeatable decision process.

The evidence, interpretation, recommendation, and next action all arrive in the same stream. The model might cite a pricing page in one paragraph, make an inference in the next, and end with a confident recommendation; and "confident" here means like "I'm 100% sure about this, trust me!" (dangerous!)

Then the session ends.

The next company gets researched in a slightly different way. The criteria move. The structure changes. The previous judgment disappears into a conversation history. Comparing two companies becomes harder than researching either one.

I did not need more research.

I needed a way to make the judgment persistent, comparable, and reviewable before I acted on it.

That became Growth Signal OS.

The goal was not to automate outreach. It was to make the decision before outreach more inspectable.

The job I was actually trying to do

The original job was simple:

Find B2B SaaS companies where I might have a credible reason to start a conversation.

Not every company with an AI feature.

Not every company with a pricing page.

Not every company that matched a generic ideal customer profile.

I was looking for a specific kind of signal: a visible product, growth, monetization, or workflow problem where my experience might be relevant.

That sounds like prospecting, but I did not want a prospecting bot. Those optimize for volume; I wanted to optimize for judgment. So the filter was narrower than "companies with an AI feature" or "companies with a pricing page" or "companies matching a generic ICP", and wider than a keyword rule could describe.

The questions were closer to whether there was a real problem here or I was inventing one, what public evidence supported it, whether the issue was important enough for someone inside the company to care, whether it connected to work I could credibly help with, what would make the hypothesis wrong, and whether there was a useful next action or the company should be left alone.

Those questions require interpretation.

But interpretation alone was not enough. The process also needed structure.

The first design decision: separate evidence from judgment

The system begins with public company research.

That might include a homepage, product pages, pricing, documentation, job descriptions, recent product communication, and other public material.

AI is useful here because the information is messy. The same underlying signal can appear in different language across different pages. A model can connect those fragments faster than a rigid scraper or keyword rule.

But the output cannot remain a free-form essay.

The research is converted into a structured evidence packet.

Each signal needs a type, a source, a description, and an interpretation. The system distinguishes what was observed from what was inferred. It also records uncertainty rather than hiding it inside polished prose.

That distinction matters.

"Pricing is usage based" is an observation. "The pricing model may discourage early adoption" is an interpretation. The second may be useful, but it is not the same kind of claim.

Once both are stored as text in a chat, the boundary becomes easy to forget. Once they occupy different fields in a schema, the system is forced to preserve the difference.

The operating loop

The workflow became this: public company research becomes a structured evidence packet, the packet passes schema validation and lands in persistent storage, a deterministic score and advisory angles get computed from it, and the record shows up in a human action queue where I make a decision and, later, manually record the outcome.

The system does not send messages, does not decide that a company should be contacted, and does not claim that a score is truth. It prepares a decision surface. That is a much smaller promise than autonomous sales, and a more useful one for the problem I was trying to solve.

Where AI belongs

AI handles the parts of the workflow that require interpretation.

It can synthesize information across public sources, identify product and commercial signals, connect signals that may belong to the same underlying problem, propose a diagnostic hypothesis, suggest an advisory angle, explain uncertainty, and identify what additional evidence would strengthen or weaken the case.

These are fuzzy tasks.

The inputs are inconsistent. The categories overlap. The same observation can mean different things depending on the company.

Trying to solve all of that with deterministic rules would create a brittle system that looks precise while missing the point.

But giving the model control of the entire workflow would create the opposite problem. It would be flexible, persuasive, and difficult to inspect.

So the model does not own the whole process.

It owns the parts where judgment is useful.

What remains deterministic

Once the research packet exists, ordinary software takes over.

Schemas validate that required fields are present.

Score bands apply consistently.

Records are persisted in a relational database.

Views calculate and expose the same fields for every company.

The action queue follows explicit states.

The dashboard reads stored data rather than regenerating an answer every time it loads.

This distinction is important.

The model may help decide whether a signal is strong, but it does not get to silently change what "strong" means for every company. It may propose an advisory angle, but it does not get to skip validation, overwrite the evidence, or contact someone.

The system uses TypeScript for the command-line workflow, Zod for validation, Supabase and Postgres for persistence, SQL views for derived information, and a React and Vite interface for review.

There is no embedded model call inside the application runtime today.

The AI work happens around the system through the tools I use to research, reason, and prepare structured records. The application then makes that work persistent and inspectable.

That may sound less impressive than saying I built an autonomous agent.

It is also more accurate.

The score is deterministic. The evidence is not.

This was one of the most useful tensions in the design.

A score can create the feeling of objectivity.

Put a number next to a company and suddenly the judgment looks scientific.

But a deterministic score is only as reliable as the evidence entering it.

The calculation can be consistent while the inputs remain debatable.

So I do not treat the score as a verdict.

I treat it as compression.

It helps compare records, sort the queue, and expose disagreements. It gives me a reason to ask why one company ranked above another.

The real value is not the number itself.

The value is that the evidence and assumptions remain available underneath it.

A good decision system should make it easier to challenge the score, not merely easier to believe it.

The human action queue

The final decision remains mine. I review the evidence, decide whether the problem is real, decide whether the proposed angle fits my experience, decide whether there is a respectful reason to contact someone, and then decide what to do next.

Sometimes the answer is outreach. Sometimes it is more research, or revisiting the company later, or doing nothing.

That last state matters.

A system designed around activity will always find another action.

A system designed around judgment needs to make inaction legitimate.

This is another reason I did not want autonomous outreach. The purpose of the workflow is not to produce messages. It is to improve the quality of the decision that precedes a message.

Making the process inspectable

"Inspectable" can sound abstract, so I mean something specific.

For any company in the system, I should be able to reconstruct what public information was reviewed, which signals were extracted, which parts were observed and which were inferred, how the structured record passed validation, how the score was produced, what advisory angle was proposed, what I decided to do, and what happened afterward.

The system is still small.

But the shape matters.

A chat gives me an answer.

An inspectable loop gives me an answer, its inputs, its state, and the opportunity to correct it.

That is the difference between using AI as a clever conversation partner and embedding it inside an operating process.

What did not work cleanly

Research quality was inconsistent. Some company pages contain clear information; others are vague, outdated, or written almost entirely in marketing language. A model can still produce a coherent interpretation from those, but coherence is not evidence.

Signal inflation crept in next. When the system is looking for opportunities, every observation starts to feel meaningful: a new pricing tier, a job posting, a product launch, a sentence in a blog post. Often they are not.

Then there was false precision. Structured fields and scores improve comparability, but they can make weak judgment look stronger than it is; the interface can appear more mature than the evidence underneath.

And there is still a real gap between a good hypothesis and a real outcome. A strong advisory angle does not mean the company agrees with the diagnosis, a thoughtful message does not mean anyone responds, and a response does not prove that the original scoring model was correct.

The current system records outcomes manually, but the operating volume is still too small to claim that it has learned what creates business results.

That limitation is important.

What the system proves

Growth Signal OS shows that I can turn an AI-assisted research process into a structured product system.

It has explicit data models, schema validation, persistent records, deterministic score bands, derived views, a review interface, an action queue, a place to record outcomes, and automated checks around the application.

It also shows a design preference I keep returning to.

Use models where judgment helps.

Use software where consistency matters.

Keep humans responsible for consequential action.

What it does not prove

It does not prove autonomous prospecting, reliable revenue generation, that the scoring model generalizes across a large market, or enterprise readiness. There is no production monitoring yet, no multi-user permissions, no autonomous ingestion, and no mature evaluation suite.

The dataset is still small.

The business impact is not established.

Calling it an autonomous growth engine would be easy.

It would also be wrong.

What I would build next

The next version should focus less on adding features and more on evaluating the judgment already inside the system.

I would start with a small golden set of company cases carrying expected evidence quality, risk flags, and decision outcomes, and track model and prompt versions against it so I can tell when behavior actually changes.

From there I would score the quality of sources rather than only the apparent strength of a signal, measure how often I correct the proposed diagnosis or angle, treat rejected recommendations as usefully as accepted ones, instrument cost and latency, and sharpen the outcome taxonomy so the system can tell no response apart from weak relevance, wrong timing, wrong contact, and a genuinely incorrect hypothesis.

The goal would not be to make the system more autonomous.

The goal would be to know more clearly when its judgment deserves trust.

The broader lesson

AI made the research faster.

That was not the most important improvement.

The real improvement came from everything around the model: the schema that separated evidence from inference, the validation that rejected incomplete records, the database that preserved decisions, the deterministic logic that made comparison possible, the action queue that kept me in control, and the outcome record that made learning possible at all.

The model generated intelligence.

The system made that intelligence usable.

I think this is where many AI experiments get stuck.

A team proves that a model can answer the question.

But the answer is not connected to state, permissions, validation, action, or feedback. It remains a demo inside a chat window.

The harder work begins after the first useful answer: deciding what to store, what to validate, what to make reversible, what to keep human, what evidence would earn more trust in the system, and what outcome tells you whether it actually helped.

Growth Signal OS is still an early system.

But it changed the way I think about building with AI.

I did not need an autonomous sales bot.

I needed a better decision loop.

And the most important thing the system produced was not another recommendation.

It was a way to inspect the judgment before acting on it.

This is the same discipline behind the Diagnostic I run now. Two sessions. The first separates real evidence from interpretation. The second turns that evidence into a hypothesis you can actually check, not a score you're asked to trust.