Federalism is accredited with averting large scale ethnic conflict in South Africa and Ethiopia in the 1990s, as both countries were spiralling towards a full scale civil war. Federal settlements provided a way out in both. While the overthrow of the centralist military regime gave Ethiopian ethnic federalism political legitimacy, in South Africa apartheid regime’s policies of entrenching tribal identities through the use of ‘homelands’ had brought a complicating element to the new federal start. Nonetheless, what united both countries was widespread support for the new federal constitutions recognising diversity and promising a new future. Twenty years later, in both countries there seems to be widespread disillusionment with their constitutions. The paper’s main argument is federal constitutions cannot deliver on their promises unless certain uncodified structural factors are taken into account. For both cases, these tend to be factors like infrastructural limitations, deficiencies in trained personnel, and dominant nation-wide parties.