The use of excavated earth is a circular and carbon neutral solution gaining growing interest. However, its high variability in properties makes it difficult to master as raw materials for construction. The objective of this paper is to highlight the key parameters allowing overcoming the influence of the variability of soil composition on the earth-based concrete properties. We show that the solid volume fraction of clay paste is the key parameter allowing predicting the fresh state properties of the clay binder, whatever the constitutive clay type, whereas the clay activity and the dry density of the material control the compressive strength of the material. These promising results, validated on sludge, pave the way to optimized mix design strategies for robust earth concrete products.