If you work in the downtown core, stroll over to Nathan Phillips Square, City Hall on your lunch break and check out the farmers' market there - it's on every Wednesday from 10:00 a.m. to 2:30 p.m. until the Wednesday before Thanksgiving.
Today I bought a monstrously huge bouquet of basil ($3!), serrano chiles, and purple potatoes. Peaches are just about over, but the gourds are coming in. The tomato baskets looked good.
Of course, the selection isn't as varied as the north building of St. Lawrence Market on Saturdays, and you certainly won't find obscure heirloom stuff, but it's a good place to get local fresh veggies and fruits directly from the hard-working farmers.