Is it bad to constantly drag chiliplants outdoors to indoors etc.?

I have some chilis growing and I've now taken my chilis outside at midday and inside when it goes under 8c. Is it bad to change the place all the time? I could also is put some haze on them when it gets colder. (it's warmer)