On the Maryland side of the Potomac you can walk from Harpers Ferry all the way up to Shepherdstown or further courtesy of the C&O Canal. On the West Virginia side things are a little more limited. But you can walk along the river a way. Which is what I did here, walking until I reached an old rubble dam usually refered to as Dam # 3 (but originally apparently called the Armory Dam).
As you can see it makes for an impressive panoramic view of the river.