It's certainly possible in terms of a large-scale contribution, but I'm not sure it's as simple as pinpointing his addition specifically. It's tough. Though he didn't get any solo writing credits until season 4, he joined in Season 3, which is the best marriage of comedy and heart the show has to offer. If he has as much as influence as you believe, then the show wouldn't have been able to reach those heights without his presence being added to the writers' room.
Plus his first solo credit, "New Kid on the Block" absolutely hits that sweet spot of balance, IMO. You've got the very well-handled and emotionally grounded A-line of Bart and the neighbor, and then the ridiculous B-line of Homer and all-you-can-eat (of which the courtroom scene is gold. "That could've been me!").
I was curious as to why there was such a shift in the series, so I did the lightest of digging (see: wikisimpsons). In crime, the saying is always "follow the money." My application of that as it pertains to tv shows is "follow the writers."
That's a serious mass exodus of talent and influence leaving. Aside from the showrunners departure, of note is Sam Simon:
Now, how much of those "creative difficulties" were related to the direction the show was flirting with in season 4? Who knows. Maybe someone who's more up-to-speed on their Simpsons lore (you, perhaps?) can answer that. But with such a large chunk of the original group leaving the show, it's tough for me to pinpoint Conan's tenure as the eventual beginning of the end. And if he did have that much influence on the show's sensibilities, then hey, no show lasts forever, and it's worth it for the peaks of Season 3 and 4. I'll still take that trade-off.