The primary challenge in modern data science is ensuring data quality rather than merely managing large volumes of data.

“but the thing is is that I think the biggest challenge when it comes to data isn't so much about volume but more about quality”

He believes that the industry often mistakenly believes high-quality data is achieved through scale, whereas it actually requires deep, subjective human judgment and sophisticated technology.

“People don't understand what quality even means in the space. They think you can just throw bodies at a problem and get good data. That's completely wrong.”

For improving retrieval-augmented generation systems, refining data preparation is more impactful than selecting a specific vector database.

“I would say this like in the a lot of the companies that I have seen that's like the biggest performance in their rack solutions coming from like better data preparations not agonizing over like what better databases to use”