The adoption of machine learning (ML) components in software systems raises new engineering challenges. In particular, the inherent uncertainty regarding functional suitability and the operation environment makes architecture evaluation and trade-off analysis difficult. We propose a software architecture evaluation method called Modeling Uncertainty During Design (MUDD) that explicitly models the uncertainty associated to ML components and evaluates how it propagates through a system. The method is based on Bayesian networks, which enable both qualitative and quantitative assessments of software architectures. In particular, the method supports reasoning over how architectural patterns can mitigate uncertainty and enables comparison of different architectures focused on the interplay between ML and classical software components. While domain-agnostic and suitable for any system where uncertainty plays a central role, we validate our approach using as example a perception system for autonomous driving. For this system we empirically demonstrate that a component-based design is over 10% more resilient to uncertainty than an end-to-end design. Moreover, we bring empirically evidence that architecture design patterns can help to significantly decrease the uncertainty associated to ML components.