We entertain the possibility of pervasive factors that are not common across two (or more) groups of securities. We propose and implement a general procedure to estimate the space spanned by common and group-specific pervasive factors. In our empirical analysis, we study the factor structure of excess returns on stocks traded on the NYSE and Nasdaq using our methodology. We find that there are only two common pervasive factors that govern the returns for both NYSE and Nasdaq. At the same time, the NYSE and Nasdaq each have one more group-specific factor that is not the same across the two exchanges. Our results point to the absence of complete similarity between the factors driving the returns on these exchanges.