Good idea i reckon, I personally think should loose prestige anyway if your wall is taken down because you have lost a part of your city, Which has made you weaker. You should be able to take down your city wall i dont see why not becuase when you begin people build traps and when you have enough money to finally get the lvl 10 turrent upgrade you are stuck with all traps,