I personally think your choice for color palette is perfect ~ brown and ivory is such a beautiful compliment to one another and if you throw in either a gold or yellow accent, it's perfection. Using sunflowers is such a great choice, I think they would pop so beautifully in the color scheme. Yanno I've noticed that brown is a love it or hate it color (i.e. an earlier comment on this thread), and of course everyone is entitled to their opinion, but if you love it (as do I) then go for it -- your opinion and preference is all that really matters, this is your day, your wedding, your celebration!
Now I do have to agree with the other gals, I wouldn't worry much on the white chairs vs. your gorgeous ivory dress. Girl, everyone's gonna be so taken aback by the beautiful setting and what a knockout you're gonna be in your uber steal of a dress find, that your chairs could be flourescent orange with purple polka dots and your guests would think it was heavenly
If you really want to draw away the attention from the white and tie in the chair color with your motiff, then maybe look into accenting the chairs with sashes in one of your colors. Last night I found a thread on here with a link to a great chair sash site that has fantastic prices, and I do recall seeing some beautiful gold, yellow, ivory or brown chair sashes in either organza or satin. Here's that site: wedding linens direct home page
But again, I think your choices are impeccable! I am the epitome of a warm, earthy tones kinda girl, so I personally can't wait to see how it all turns out ~ which I know will be fabulous I'm sure!!!