The conversation between Walt and Donald Margolis is an important one for me in developing the Walt/Jesse father/son motif. They talk about their daughters (though Holly is just born), but the conversation centers more around sticking by family through thick and thin. Despite Walt's constant manipulation and criticism of Jesse, I think there is an argument to be made that he generally does stick by him--that is, until "Ozymandias".
Otherwise, the location isn't very remarkable. The building looks a bit prison-like. I don't recall any windows. Despite this, it feels like a place that would be a bit more main-drag, perhaps off Central as opposed to Yale.
Burt's appears in a montage that is perhaps my favorite of all in the series. The meth cook pieces are of course legendary, and the missing poster sequence from "Grilled" has perhaps one of the best overall songs ("Red Moon" by The Walkmen). But the meth-selling montage from "Breakage" is set to the infectious, celebratory, and culturally-significant "The Peanut Vendor".
The bar is very close to the Simms Bldg., aka DEA headquarters (hence, doubtful that even someone as goofy as Combo would sell there). It's close to a tattoo shop as well, so we saw some inked-up folks lurking. Burt's really looks rough during the day; it almost looks abandoned. When we stopped by again at night, neither the "U" nor the "R" were lit.