I thought it would be a piece of cake to make the drive. If I knew I would've flown. Driving through Baltimore and DC is pure hell.
You should have taken US 301 from near Wilmington to north of Richmond, or at least driven around the east side of the Baltimore Beltway and picked up US 301 west of Annapolis (via I-97 and MD 3).
I live in the DC area, I can tell you all about the traffic on I-95. It is particularly awful between a little south of the DC Beltway and Fredericksburg.
EDIT: I assume you'll be heading back to New Jersey today. Just because it is the weekend does NOT mean I-95 will be OK between Fredericksburg and DC. (Between DC and Baltimore should be OK.) Take a route that avoids I-95 in northern Virginia, like the route I mentioned above or, depending on where you drove from in New Jersey, a further west route through central Virginia that eventually allows you connect up with Interstate 81 and then Interstate 78 east of Harrisburg.
Also, if you attended Game 1 and/or Game 2, my condolences. I was at Game 2 myself last night (after driving down from the DC area Friday late morning/early to mid-afternoon) and that was awful enough.