While not being evil himself, the man is indirectly responsible for an enormous amount of suffering.
When we think about villains, we usually tend to imagine malicious sociopathic characters, filled with dark desires like thirst for power and blood, greed or anger. And naturally, judging by this definition, Hiruzen Sarutobi — also known as The Third Hokage — can by no means be called a villain. A benevolent and virtuous person he represents everything opposed to evil and ultimately loses his own life in a duel against Orochimaru — the archvillain of the Naruto's first season.
However he does suffer from a couple of very particular shortcomings that continually bring multiple horrendous calamities onto the people of the Hidden Leaf village. With some of them not only taking many innocent lives but also leading to very dire consequences further down the story — many years after the event.
The decimation of the Uchiha clan, the destruction of Konoha by Pain and the disastrous incident during the Chunin exams — all of that happened on The Third Hokage's watch and yet nobody seems to hold him accountable for it.
While at the first glance Hiruzen Sarutobi might give an impression of a very wise and strong leader, upon further examination we can clearly see that his competence as the ruler of the Hidden Leaf village is — at best — greatly exaggerated. His failures begin with the appointment of the village council, which he composes of three highly intellectual and yet extremely uncompassionate and strict-to-the-point-of-absurdity individuals, whose views and approaches towards various problems are absurdly inflexible.
These people continuously fail The Third Hokage forcing him to make the wrong judgment in regards to many delicate issues. And, as if that alone was not enough, they also at several occasions personally fuel internal conflicts — the biggest of which notoriously results in the massacre of the Uchiha clan by Itachi.
The trusting nature of Sarutobi, his inability to see through people as well as his total lack of grasp of the concept of even the most basic political intrigues make him absolutely unfitting for the role of Hokage.
But, even when various disasters start reigning down upon the village, he still is reluctant to release the post which culminates in Orochimaru's attack on Konoha dealing a huge blow to all the villagers and Sarutobi losing his life.
On top of his failures as a leader, The Third Hokage also commits several giant personal mistakes like forbidding Kushina from raising Naruto or letting Orochimaru slip through his fingers when he was first caught. Ultimately this paints us a picture of a man who was entrusted with a very important duty but ended up making a huge mess out of it forcing the people around him to suffer greatly.
He is by no means evil and yet his incompetence does bring about numerous disasters making him indirectly responsible for such a long list of mishaps that he can arguably be put on the same pedestal as the 'proper' main villains of the franchise.