Non-work travel in the proportion of the structure of travel is on the rise. Structural equation modeling (SEM) was adopted to analyze the influence mechanism of residents’ non-work activity and travel behavior based on the Shanghai Household Travel Survey. The results of SEM show that socio-demographic characteristics and neighborhood features both affect residents’ non-work activity and travel, but neighborhood features have stronger influence than socio-demographic characteristics. The most significant characteristic in individual is the employment situation. The most significant characteristic in household is the number of automobiles. And the most significant characteristic in neighborhood is the coverage of neighborhood services.