I build synthetic respondent panels myself, and I document exactly where they break. Because I don't sell the data, I can be the honest check a vendor structurally can't be.
The averages look fine. Variance collapse hides in the cuts you actually care about: the niche segments synthetic sample was supposed to help you reach.
Everyone sounds a little too happy. Language models are trained to be agreeable. Synthetic respondents inflate positivity in a predictable direction.
The vendor grades their own homework. The people who sold you the boost have no incentive to tell you which cuts you can't defend in a stakeholder meeting.
Every study below is my own synthetic panel, validated against real data, with every limitation published. The failures are the point.
Synthetic gamers predicted the shape of real survey results, then missed the numbers by 1.6 NPS points. The reason: language models can't do irony on their own.
Read the study → Study 02 · FeaturedAn eNPS question collapsed onto a handful of scores. Better grounding bought the variance back, run over run: 82% to 78% to 66%. It still never fully returned.
Read the study → Study 03 · Prediction150 synthetic researcher personas, four independent anchors, one public prediction about the industry's own report. The verdict lands when GRIT does.
Read the study →"Trust synthetic data for the direction of a relationship. Never trust it for the exact numbers. Use it to pre-test, de-risk, and find patterns. Then validate."
I'm Arnold Santiago. I've spent my career on the supplier side: running online panels, managing fraud detection, and executing studies from MaxDiff to political polling for some of the most demanding insights teams in the business.
My job, for over a decade, has been telling clients which respondent data to trust and which is noise. Synthetic data is the newest version of that same question. I'm not here to sell you on it or scare you off it. I'm here to check it.
Hand me one dataset. In about a week you get a plain-language memo on where it holds, where it's distorted, and which cuts you can defend. Plus a call to talk it through.