If you're ready to dig a little deeper than the typical Paris guidebook and you don't have the time or the dough for one of those incredibly pricey personal tours of lesser known venues, here are a couple of resources you may find handy. The first is a series of 31 Balades du Patrimoine available on-line from the city of Paris. (You can also get these in an attractive print format but honestly, I can't remember now where I picked them up.) Each guide follows a certain theme -- horses and knights, church bells, fountains, stained glass windows, for example -- and provides a map for a walking tour along with explanations of the various themed sights along the route. (Fair warning: the guides are in French only. If you're like me and your reading French kicks the stuffing out of your speaking French, you shouldn't have a problem.)
The other is a joint effort of the Pavillon d'Arsenal, an information center focused on the architecture of Paris, and the RATP, Paris's own public transport authority. Working together, the team has created a series of guides, Archi-Bus, that provide interesting details about contemporary architecture that can be seen along eight public transport lines. I discovered the series after picking up a brochure on the bus but it may be a safer bet to download them yourself. This one is also in French only.
Now get out there and see you some sights.