The JR Pass is only really going to be valuable for round-trip travel between Tokyo and Kansai and if you have long side trips planned in between. Since Osaka, Kyoto, and Nara are all close together and it sounds like you’re going to spend more than one week in that area, I’d recommend just buying individual tickets instead of getting the pass because a two-week JR Pass is definitely not worth it for what you’re planning.
So my advice would be to buy Suica or Pasmo fare cards once you get to Tokyo. The ticket machines that dispense them have English menus and you can reload them at the same machines whenever you need to. Use them for rail tavel during your time in Tokyo. Shinjuku is a major hub; if you can’t get somewhere with one subway transfer, then the JR Yamanote which loops through all the major neighborhoods can. Then when you’re ready to go to Kansai, buy Shinkansen tickets from the Green Window at Tokyo or Shinagawa Station.
Since you’ll have a baby to walk around with, a single day in Nara might be enough, since most of the interesting stuff is all concentrated in the park. The deer might actually be scary for your child, since they can get aggressive (and bitey!) once they detect that you have rice crackers. Osaka and Kyoto are only half an hour apart by local train, so whether you want to do a lot of back-and-forth will determine if you want to pick one city to stay in or if you want to move after a few days.
As for stuff to do… I guess it depends on what stuff interests you. Cheapo has an article with things to do with kids. Ueno Zoo in Tokyo and Kaiyukan Aquarium in Osaka would probably be good.