The ISB produces a public repository for spectral ion libraries called SWATHAtlas (www.swathatlas.org) and the libraries available have been typically used to analyze the SWATH type mass spectrometry experiments using data-independent acquisition (DIA) methods. For confidence in protein identification and reporting quantitative results, the characteristics of a reference ion library are important when processing SWATH data. In this study, we provide an extensive demonstration of the effect of different libraries onto the performance of SWATH analysis tools and reported results that incorporate different chromatography flow rates, different mass spectrometry instruments and the most widely used SWATH analysis software tools. For consistent assessment of reference ion libraries, we developed DIA-Lib QC, a library assessment tool to calculate the metrics of library correctness and its completeness, which ensures accurate library-based SWATH analysis. With our benchmarking hybrid datasets, we evaluated the characteristics and performance of different SWATH-MS tools such as OpenSWATH, PeakView, Skyline, Spectronaut, DIA-Umpire, and DISCO based on the accuracy of identification, the precision of quantitation, quantitative sensitivity, and relative performance of these applications. Conclusions include tools perform with higher fidelity with targeted reference libraries, library-based tools provide higher agreement in overall identifications with targeted libraries, reliable quantification performance, and robustness for label-free quantitative proteomics.