Sampling-based algorithms have dramatically improved the state of the art in robotic motion planning. However, they make restrictive assumptions that limit their applicability to manipulators operating in uncontrolled and partially unknown environments. This work describes how one of these assumptions - that the world is perfectly known - can be removed. We propose a utility-guided roadmap planner that incorporates uncertainty directly into the planning process. This enables the planner to identify configuration space paths that minimize uncertainty and, when necessary, efficiently pursue further exploration through utility-guided sensing of the workspace. Experimental results indicate that our utility-guided approach results in a robust planner even in the presence of significant error in its perception of the workspace. Furthermore, we show how the planner is able to reduce the amount of required sensing to compute a successful plan