The recovery rate of construction and demolition waste (CDW) in the European Union (EU) is at 89 % and thus high relative to other waste streams. However, the relatively high figure can be misleading because it typically does not correspond to high-value material recovery but rather “poor” levels of circularity. From a life-cycle perspective, we assess the environmental impacts and costs of 12 CDW material fractions relying on alternative pathways and treatment technologies. The results indicate important trade-offs in the transition towards the circular economy. Indeed, recycling of concrete, bricks, gypsum, and ceramics and tiles represent the best environmental performance but also the most expensive pathway. However, when shifting from landfill to recycling the total societal costs in the EU are reduced mainly due to the lower external costs. Overall, recycling CDW in the EU with advanced technologies would save about 264 kg CO2-eq t−1 with a cost of 25 EUR t−1. The maximum potential for recycling under current technology in the EU would lead to an annual total reduction of about 33 Mt. of CO2-eq using 2020 as reference year. The fractions with the highest potential for improving current waste management practices in terms of environmental improvements are concrete and bricks. The economic and non-economic barriers for realising this potential at EU level are discussed in relation to the European Green Deal and the EU’s circular economy objectives.