Laying a siege is so complex that I can't possibly fit even a brief sketch into one blog post. Since Part III of my fantasy monstrosity involves a siege, expect to hear more on this.
Armies and cities have always been popular breeding grounds for microbes. Both armies and cities bring together diverse groups of people who haven't met before -- somebody is bound to bring a new virus or bacteria to the party. Then the disease has a field day infecting all these bodies whose immune systems haven't seen it before.
Diseases spread by airborne particles, insects or mobile parasites like fleas and lice, and excrement (depending on sanitary conditions) can spread easily in these conditions.
Historically "popular" parasite/insect-related diseases: malaria, typhus, plague
Airborne diseases: smallpox, pneumonia, measles
Sanitation-related diseases: dysentery, typhoid, cholera
I've heard it said that even a modern city only has enough food on hand to feed itself for a few days. And that's with the conveniences of modern canning and freezing.
Even if a city saw this siege coming with enough advance notice to lay in stores, feeding thousands of people for weeks and months is going to rapidly become difficult. More than one description of a siege in the Hundred Years' War described the besieged city expelling parts of its population (elderly, disabled, children, sometimes women too) because they could not afford to feed them. Since the attacking army didn't need extra mouths to feed, either, these refugees generally ended up huddling against the city wall starving and dying of exposure.
The army outside must feed itself either by receiving regular shipments of supplies or by scrounging around the surrounding territory. Supply lines are natural targets for attack, of course. Scrounging in a hostile countryside for ten thousand hungry men isn't easy, either.
Among the defenders, this is obvious enough -- someone opens the city gates, kills commanding officers, poisons the wells, or otherwise gets the attackers the help they need to take the city.
Traitors among the attacking army could spread misinformation, could inform on where and when supply caravans are expected, and could also kill commanding officers. Historically, armies have been known to fall apart when nobles of significant power withdrew their men and support. This can happen for any number of reasons, from personal conflicts with other ranking nobles to sympathy for the enemy.
All these things to worry about before we've even fired the first arrow over the battlements. Which one would have the most impact on your fantasy siege?