SINGULAR VALUE VERSUS EIGENVALUE DECOMPOSITION EFFICIENCY IN COMPUTING PRINCIPAL COMPONENTS FOR DIMENSIONALITY REDUCTION OF LARGE DATASETS
DOI:
https://doi.org/10.26408/134.04Keywords:
Big Data, dimensionality reduction, principal component analysis, singular value decomposition, eigenvalue decomposition, performance, efficiencyAbstract
Principal component analysis, being one of the best techniques for dimensionality reduction, is implemented by using one of the two high-accuracy algorithms: the singular value decomposition (SVD) and eigenvalue decomposition (EVD). The EVD is generally faster than the SVD, except for datasets with fewer observations or when the observation has fewer features. Apart from cases of shallower datasets consisting of just a few hundred double-precision observations, the EVD speeds up computing principal components by at least 4.5%, whereas the average speedup in 45% widely varies from 12% to 92%. The speedup on non-shallower single-precision datasets is roughly similar, but it nonetheless makes no sense due to EVD poor accuracy while operating on numeric data with single precision. The EVD is efficient if the dataset consists of no fewer than a few hundred observations (objects) having at least three double-precision features.
References
Akter, S., Wamba, S.F., 2023, Handbook of Big Data Research Methods, Edward Elgar Publishing.
Demirbaga, Ü., Aujla, G.S., Jindal, A., Kalyon, O., 2024, Big Data Analytics. Theory, Techniques, Platforms, and Applications, Springer, Cham.
Downey, R., 2024, Computational Complexity, [in:] Computability and Complexity. Undergraduate Topics in Computer Science, Springer, Cham, pp. 159–176.
Enderton, H.B., 2011, 7 – Polynomial-Time Computability, [in:] Computability Theory, Academic Press, pp. 139–149.
Golub, G.H., Van Loan, C.F., 1996, Matrix Computations, Johns Hopkins University Press, Baltimore, MD, USA.
Härdle, W.K., Simar, L., Fengler, M.R., 2024, Principal Component Analysis, [in:] Applied Multivariate Statistical Analysis, Springer, Cham, pp. 309–345.
Holmes, M.H., 2023, Eigenvalue Problems, [in:] Introduction to Scientific Computing and Data Analysis. Texts in Computational Science and Engineering, vol. 13, Springer, Cham, pp. 129204.
Jamarani, A., Haddadi, S., Sarvizadeh, R., Kashani, M.H., Akbari, M., Moradi, S., 2024, Big Data and Predictive Analytics: A Systematic Review of Applications, Artificial Intelligence Review, vol. 57, no. 176.
Jolliffe, I.T., Cadima, J., 2016, Principal Component Analysis: A Review and Recent Developments, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, vol. 374, no. 2065.
Kontoghiorghes, E.J., 2020, Handbook of Parallel Computing and Statistics, Chapman & Hall.
Krishnan, R., Samaranayake, V.A., Jagannathan, S., 2019, A Hierarchical Dimension Reduction Approach for Big Data with Application to Fault Diagnostics, Big Data Research, vol. 18, no. 100121.
Li, B., Liu, Z., Xu, Z., Li, J., Wang, B., Song, M., 2025, Vehicle Appearance Dataset, in: Pattern Recognition and Computer Vision. PRCV 2024. Lecture Notes in Computer Science, vol. 15039, Springer, Singapore, pp. 407–421.
Li, H., 2024, Singular Value Decomposition, [in:] Machine Learning Methods, Springer, Singapore, pp. 311–336.
Meepaganithage, A., Nicolescu, M., Nicolescu, M., 2025, Enhanced Maritime Safety Through Deep Learning and Feature Selection, [in:] Advances in Visual Computing, ISVC 2024, Lecture Notes in Computer Science, vol. 15047, Springer, Cham, pp. 309–321.
Oliveira, J.J. M., Cordeiro, R.L.F., 2020, Unsupervised Dimensionality Reduction for Very Large Datasets: Are We Going to the Right Direction? Knowledge-Based Systems, vol. 196, no. 105777.
Olivieri, A.C., 2024, Principal Component Analysis, [in:] Introduction to Multivariate Calibration, Springer, Cham, pp. 71–87.
Olszewski, D., 2025, Asymmetry Index for Data and Its Verification in Dimensionality Reduction and Data Visualization, Information Sciences, vol. 689, no. 121405.
Paluszek, M., Thomas, S., 2024, Data for Machine Learning in MATLAB, [in:] MATLAB Machine Learning Recipes, Apress, Berkeley, CA, USA, pp. 21–48.
Posthoff, C., 2024, Polynomial and Exponential Complexity, [in:] Artificial Intelligence for Everyone, Springer, Cham, pp. 41–62.
Romanuke, V.V., 2018, Decision Making Criteria Hybridization for Finding Optimal Decisions’ Subset Regarding Changes of the Decision Function, Journal of Uncertain Systems, vol. 12, no. 4, pp. 279–291.
Romanuke, V.V., 2019, Generator of a Toy Dataset of Multi-Polygon Monochrome Images for Rapidly Testing and Prototyping Semantic Image Segmentation Networks, Electrical, Control and Communication Engineering, vol. 15, no. 2, pp. 54–61.
Romanuke, V.V., 2023, Speedup of the k-means Algorithm for Partitioning Large Datasets of Flat Points by a Preliminary Partition and Selecting Initial Centroids, Applied Computer Systems, vol. 28, no. 1, pp. 1–12.
Romanuke, V.V., 2024, Deep Clustering of the Traveling Salesman Problem to Parallelize its Solution, Computers & Operations Research, vol. 165, no. 106548, pp. 1–20.
Si-ahmed, I., Hamdad, L., Agonkoui, C.J., Kande, Y., Dabo-Niang, S., 2025, Principal Component Analysis of Multivariate Spatial Functional Data, Big Data Research, vol. 39, no. 100504.
Zimand, M., 2004, Chapter 3 – Polynomial Time, Nondeterministic Polynomial Time, and Exponential Time, [in:] North-Holland Mathematics Studies, vol. 196, North-Holland, The Netherlands, pp. 51–108.
Zizler, P., La Haye, R., 2024, Singular Value Decomposition, [in:] Linear Algebra in Data Science. Compact Textbooks in Mathematics, Birkhäuser, Cham, pp. 73–101.
Downloads
Published
How to Cite
Issue
Section
Categories
License
Copyright (c) 2025 Vadim Romanuke
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors retain the copyright to their work, licensing it under the Creative Commons Attribution License Attribution 4.0 International licence (CC BY 4.0) which allows articles to be re-used and re-distributed without restriction, as long as the original work is correctly cited. The author retains unlimited copyright and publishing rights.
