The Elephant Mountain hiking trails aren’t bad (though I wouldn’t call them a hike rather a short climb). If the weather is clear then there are nice views from the top; if interested you can find many trails nearby the Taipei 101 building. Do visit the 101; the observation deck is cool (been in my share of tall buildings and these elevators are sick) and I thought that giant ball (stabilizing building) was cool – overall, they give a lot of info about the architecture/construction, etc. which I liked. Also had coffee and hung out at a bucks on level 35(?) that offered a view.
The Palace Museum could work if you have an interest in Chinese antiquities with two caveats: there could be massive queues here (mainland tour groups) so be mentally prepared for that (though I am told that you could visit late afternoon or evening when less people are around); and definitely hook up a guide so that you can understand the significance and meaning of this stuff.
cheers.