The expressive capacity of physical systems employed for learning is limited by the unavoidable presence of noise in their extracted outputs. Although present in biological, analog, and quantum systems, the precise impact of noise on learning is not yet fully understood. Focusing on supervised learning, I will present a mathematical framework for evaluating the resolvable expressive capacity (REC) of general physical systems under finite sampling noise, and provide a methodology for extracting its extrema, the eigentasks. Eigentasks are a native set of functions that a given physical system can approximate with minimal error, and the construction of low-noise eigentasks from measurements provides improved performance for machine learning tasks such as classification, displaying robustness to overfitting. I will then discuss how the REC of a quantum system is limited by the fundamental theory of quantum measurement and how the latter imposes a tight upper bound on the REC of any finitely-sampled physical system. The applicability of these results in practice will be demonstrated with experiments on superconducting quantum processors. I will present analyses suggesting that correlations in the measured quantum system enhance learning capacity by reducing noise in eigentasks and discuss the implications of these results for quantum sensing.