I kind of like the idea. The I'm picturing it is that the river is pretty much the border of the town, so most of the buildings are on the defending side. There's a drawbridge across, but it's controlled from the defending side. This means that if defenders want to bring it down to move out, they run the risk of letting the attackers in.
I like it, but if the bridge was really important, only the attackers should have boats. That way, it puts more pressure on the defenders to make the decision.
Oh god, I'm going to get very carried away with this idea.