Windows in the OS X Finder–the equivalent, more or less, of Windows Explorer–normally display the title of the current folder. You can fire up a Terminal and type the following two lines to change this default:
defaults write com.apple.finder _FXShowPosixPathInTitle -bool YES killall Finder
If you change your mind later, you can change it back later with this:
defaults write com.apple.finder _FXShowPosixPathInTitle -bool NO killall FinderThere are lots of hidden customization strings of this sort in OS X. Google "defaults write com.apple.finder" for more.