On 11 June 1797, Father Fermín Francisco de Lasuén raised a cross at the Ohlone village site of Oroysom and founded Mission San José, the fourteenth of California's twenty-one Spanish missions. One of the earliest European settlements in the East Bay grew here, decades before the Gold Rush and long before anyone had heard of Silicon Valley.

The mission became one of the most prosperous in northern California, with vast herds and harvests. Under Father Narciso Durán it was also famous for music: he trained an orchestra of about thirty Ohlone musicians whose performances drew visitors from across the region.

The great earthquake of 1868 on the Hayward Fault leveled the adobe church. The building standing today is a faithful reconstruction completed in 1985, beside original mission-era quarters that survived.

The mission gave its name to the surrounding district, which in 1956 joined four neighboring towns to incorporate as the city of Fremont, the same city where Yu-kai Chou grew up, a few kilometers away near Lake Elizabeth.