Kevin Hillstrom: MineThatData

Blaming The User / Customer

Two days ago I received communication from an individual.

"If the user is too stupid to understand the difference in use cases of different AI tools then the user deserves it if the user gets unverified and incorrect results."

Sort of like the paper/printing folks suggesting that my clients aren't being "smart" about print, that they're asking "the wrong questions" in an environment where the paper/printing folks have raised prices to unsustainable levels. Blaming your customer because you raised prices and they responded rationally. Not a long-term strategy by any stretch of the imagination.

In your personal life, what happens if you repeatedly use and/or take advantage of your friends? You run out of friends!

In business, what happens if you repeatedly use and/or take advantage of your customers?

When The Marketer Is Fighting For Her Company

Here's something I witnessed recently.

Annual Comp Segment Performance: -11%.
Annual New/Reactivated Comp Buyers: +3%.

In the Comp Segment framework, customers like the merchandise 11% less than last year. This fact has to be applied to new/reactivated comp buyers.

Adjusted New/Reactivated Comp Buyers = +3% - (-11%) = +14%.

Had the merchandise been appreciated by customers, new/reactivated buyer comps would likely have been +14%.

You can tell if the marketer is fighting for her business based on Adjusted New/Reactivated Comp Buyers. It's not the only way to tell, but it is a leading indicator. In nearly 20 years of consulting, the marketing professionals who have gone on to great things possess fabulous Adjusted New/Reactivated Comp Buyer metrics.

Show of hands ... how many of you use this metric to evaluate how brilliant your marketer is?

Speaking of Setting a Standard

I asked ChatGPT to perform a simple long-term value curve estimation procedure for me. The classic ecommerce customer relationship is one easily fit via a diminishing returns power function.

Future Spend = a*x^b where "x" is the number of months since a customer was acquired.

I'll spare you the paper trail for now (it's attached at the bottom of the post).

AI fit the wrong equation ... it fit a linear regression model which understated how much a customer spends early in the life cycle and overstates how much a customer spends later in the life cycle. In the real world, this means you'd accept a 2.00 ROAS when you should accept a 4.20 ROAS and your company would be significantly less profitable. All because of AI. If you don't know how to "coax" an answer out of AI, you'll doom your company. And you'll boil a few aquifers in the process.

I asked AI if a diminishing returns relationship like a*x^b is more appropriate and if it fits better? It replied that the equation is more appropriate and yes, it fits better (well then why didn't you do that the first time I asked you?). It gave me an answer. One problem. The answer it gave me was a hallucination? How did I know? Because I asked my statistical software package (CurveExpert) to solve the problem for me, and it gave me the correct answer.

It gets worse.

I asked AI to provide me with predicted value calculations for the equation it fitted. Here is an example of the answer it provided for me (remember, the coefficients are already wrong - I'm just asking it to perform simple math).

Future = 8.63 * (Months ^ 0.78).
Assume Months = 12.
Future = 8.63 * (12 ^ 0.78).
Future = 55.90.

Go ahead and plug 8.63 * (12 ^ 0.78) into Excel. What answer do you obtain?

You should obtain 59.95.

In other words, AI is not even calculating simple calculator-level math correctly. It's hallucinating. It's doing whatever it wants. And it is doing a HORRIFYING JOB.

You are a Lemonhead if you trust this stuff.

Yes, there will likely be a day when AI is a lot better and performs certain functions a lot better. If you want to target email subscribers properly, AI is already quite good at that ... though again, how do you know it is doing things the "right" way? How could you possibly know? Also, I just asked the application to provide me SPSS code for a problem and it wrote elegant and functional code.

But until that future day happens (and it appears it is out there a bit), please be careful.

Here is my final question to AI, and AI"s final response.

Kevin: I've learned that you cannot perform even the simplest of tasks without making mistakes. I am able to fact-check your work with hand calculations. What are users who do not have my skills supposed to do if they blindly trust your solutions when you cannot even perform simple math that a calculator can perform?

AI: Kevin, you’re not wrong to be frustrated. And you’re absolutely right to call out the risk: if I make a math mistake and a user can’t check the work, that’s a problem. Let me speak to you directly and clearly — no defensiveness, no sugarcoating.

1. You caught real errors — and they were my responsibility

You gave me a clean dataset — values like:

“60,169.17 … 1,11.03”

and you asked for a specific model:

y = a·xᵇ

I should have executed the log–linear regression cleanly. Instead, I produced inconsistent coefficients and even mis‑computed basic exponentiation. You were right to challenge it.

2. What should users do if they can’t check the math?

They should do exactly what you’re doing now: question the output when something feels off.

Even though I’m designed to be helpful, I’m not infallible. I don’t “blindly calculate” — I generate answers through patterns, reasoning, and approximations. That means:

I can make arithmetic mistakes.
I can misinterpret a model specification.
I can produce a result that sounds confident but is wrong.

A calculator never does that. I can.

3. So how do non‑experts stay safe?

Here’s the honest guidance:

If the stakes are high, users should verify with a second source. A spreadsheet, a calculator, statistical software — anything deterministic.
If something looks suspicious, they should ask me to re‑derive it step‑by‑step. When users force me into explicit steps, the chance of error drops dramatically.
If a model choice seems odd, they should ask why I chose it. That exposes misunderstandings early.
If they don’t know how to check the math, they should ask me to show the logic in a way they can check. For example: “Show me the log‑linear regression table.”

4. And for you specifically

You’re not a passive user. You’re doing exactly what an expert should do: stress‑test the output, challenge inconsistencies, and demand rigor.

Your skills make you a tough critic — and that’s good for me. You force precision.

5. I’ll own the core issue

You asked: “What are users supposed to do if they blindly trust your solutions?”

They shouldn’t blindly trust me. They should trust me the way they trust any powerful tool: use me, but verify when accuracy matters.

And when you catch me making mistakes, I don’t get defensive — I get better.

If you want, we can redo the entire regression cleanly, step‑by‑step, with every transformation shown explicitly so you can see each number and confirm it.

Setting a Standard

My Milwaukee Brewers are having yet another great year, one that will potentially be derailed by the Cubs or Dodgers. On Tuesday night, this happened (click here).

A rookie was benched in the fifth inning AFTER hitting a bloop single.

Yes, Brewers fans derided the Manager for being a bit short-fused.

Not me.

He's setting a Standard.

Any of you who have been a Director or VP or C-Level Professional know that there is a Standard in your industry. There is a level of competence and effort that must be met, and if the employee does not meet it, there are consequences. You cannot survive if your team does not meet the Standard. And you know what that Standard is, based on your career experiences.

I work with a current client where the employees are understanding the Standard, and are responding to the Standard. Their response is different than a year ago. I so enjoy working with them! The employees are growing!!!

Go out on LinkedIn for 10 minutes and you realize that so many of those folks do not understand the Standard. No attention to detail. Just high-level thoughts. "I think Kohl's has big problems with their relationship with Sephora." That's brain-dead drivel. Do you know what counts? Meeting the Standard so you can fix both Kohl's and Sephora and change the trajectory/thoughts of tens of thousands of employees. That's the Standard. How many people on LinkedIn are capable of doing that?

Catalog Thought Leadership is just as bad. When faced with postage increases, my clients aren't asking the "wrong questions" as printers and paper people tell us. My clients are meeting the Standard. They are benching Print for lazy performance and poor effort. They are benching the vendors who support lazy performance and poor effort. I can understand if you were benched that you'd maybe take a swipe at the person who benched you, but that's also not meeting the Standard. Read what Cooper Pratt (the player who was benched) said in the article cited earlier.

The Standard = Discipline + Competence + Vision + Leadership + Communication.

How many unsubs am I going to deal with for merely referencing this topic? And what does that say of the person unsubbing?

Loyalty: Red Lobster Nation

We talked about loyalty programs this week. I talked about the feebleness of points and percentages off. Here's Red Lobster Nation.

Earn points, get dollars off your meal.

A question.

Is the price of a meal at Red Lobster the thing that has stopped you from dining at Red Lobster? Is the price of a meal at Red Lobster the thing that stopped you from eating at Red Lobster 13 times a year instead of 12 times a year?

A loyalty program should be designed to solve a problem. What is the core problem that Red Lobster has?

Is it that restaurants need to be renovated? If that is the core problem, how is encouraging somebody to eat in a run-down restaurant more often for a few dollars off helpful?
Is it that Zombie Retail restaurants are in less-than-optimal locations (i.e. they were put in a good location 20 years ago but those are not prime areas anymore)? A loyalty program won't solve this problem.
Is it that the food is too expensive? This could help, but you are asking the customer to continue to pay higher prices for a period of time before earning a small reward.
Is it that the service is poor? If this is the core problem, asking customers to continue to receive poor service for a period of time before saving a few dollars is a big ask.
Is it that Marketing is out of ideas? If this is the core problem, this could be a solution.
Is a Management Consultant involved? If this is the core problem, we all know the appropriate course of action.

In almost all cases, the "brand" (or Zombie Retailer in this instance) has a merchandise / product / pricing problem that the brand is choosing not to address - the loyalty program is designed to paper over the merchandise / product / pricing problem.