Two other places that might be worth a visit:
(taking care to leave torches and pitchforks behind)
The National Cryptologic Museum is located next to the National Security Agency HQ. It's really not that far away.
https://www.nsa.gov/about/cryptologic_heritage/museum/
The B&O Train Museum is a must-see stop for anyone interested in railroads - http://www.borail.org/Collections.aspx
I remember spending a fun afternoon several years ago (okay, so it's been over 15 years now...) just riding the water taxi around the harbor, getting off and wandering around Fells Point as well.
Following the trains & NANOG theme - one can visit the site of the Howard Street Tunnel fire:
http://en.wikipedia.org/wiki/Howard_Street_Tunnel_fire
or as NANOG saw it:
https://www.nanog.org/mailinglist/mailarchives/old_archive/2001-07/msg00351.html
loved this follow up:
https://www.nanog.org/mailinglist/mailarchives/old_archive/2001-07/msg00370.html
(I’m amazed how well the list is archived - this was over 13 years ago.)
I was downtown at the time and they were telling every one that "the air was toxic, close your windows"…and me stuck in traffic in my convertible with a worn out (torn) roof.