Systems and applications rely heavily on data, which makes data quality a critical factor for their function. In turn, low quality data can be incredibly costly and disruptive, leading to loss of revenue, incorrect conclusions, and misguided policy decisions. Improving data quality is far more than purging datasets of errors; it is more important to improve the processes that produce the data, to collect good data sources that are used for generating the data, and to truly understand the quality of the data. Therefore, the objective of this thesis is to improve and understand data quality from the above aspects. First, we develop two efficient and effective tools, DataXRay and QFix, that are able to diagnose systematic errors in general data extraction systems and relational data systems respectively. Second, we design a recommendation system, Midas, that focuses on identifying high quality data sources for augmenting knowledge bases. Third, we implement an explaining system, Explain3D, which explains the disagreements in disjoint datasets.