Reddit reviews How to Break Software: A Practical Guide to Testing W/CD

> What kind of testing do you use for data analysis (ie when you don't know what the result should be)? Do you go so far as to make fake data where you know the answer?

Yes. You craft fake data which will yield a known outcome. Then you run the analysis. Then you compare the results of analysis against the expected outcome. This can become tedious, so you automate the testing with scripts or frameworks for that purpose. In general:

• Craft data sets you know will cause analysis to succeed.
  
• Craft data sets you know will cause analysis to fail.
  
• Craft data sets you believe will test the boundaries of analysis.
  
  Perhaps you are further along in your understanding, but it sounds like you may still be trying to wrap your mind around the entire woolly concept of testing. If that is the case, you might benefit from How to Break Software. Unfortunately, that book is not about data analysis, specifically, but testing analytical code is a special case of testing other kinds of code.