Common Challenges with a growing Drupal site and how to handle them

Presenter(s): 

Building a site in Drupal is easy but as the site grows in the terms of complexity and sizes its maintenance becomes equally challenging. Some of the most common challenges that we have with most of the growing drupal sites are

1. Maintaining the consistent level of performance, even if the number of users increase on the site.
2. Drupal core requires patching and patches are often conflicting and have to be reapplied with each drupal upgrade,
3. Original patches are often unmaintained.
4. Sites stagnate, running old, insecure versions of Drupal core because updating is too difficult.
5. QA of the site becomes increasingly difficult. Regressing the whole site for every feature becomes difficult on every release and QA time and efforts keep on increasing with every release
6. Code Management in drupal is a challenge because it is Open Source and cant be compiled into libraries unlike Java and Dotnet.
7. How to identify which module should be used when you have lot of options to use for.
8. Contrib modules use lot of memory and as you increase features, you need more RAM or lot of optimization
9. Manual updates and deployment: Slow deployment, non atomic and difficult rollbacks.
10. Configuration testing on a staging severs.
11. Optimizing searches using Apache Solr.

Because of these common problems/Challenges the maintenance and QA of these large Drupal sites become difficult and as a result of it users prefer to migrate their sites on to other CMS like Joomla, Wordpress, etc. Today 12% (Please check for these numbers) of websites are being created in Wordpress as compared to only 2% of Drupal. In this session we will try to figure out the reason why these challenges arise with the growing size of the drupal sites and can something be done to handle them at the initial stages rather than being concerned about them when site already becomes big.

Questions answered by this session:
1.How can we ensure that the performance and functionality of the drupal site does not get affected as it gradually grows in size and features.
2.What are the tools to trace all the hooks used on a page?
3.How can we easily maintain the drupal site?
4.How useful it is to use Selenium to automate the testing cycle?
5.How to effectively Manage the open source code?

Examples:
I presented "Scaling Drupal with Confidence- Tweentribune case study" in Drupal Camp Boston 2010 which covered the challenges our team faced and the solutions we found in scaling Tweentribune to around 3 million page views a month. http://boston2010.design4drupal.org/sessions/scaling-drupal-confidence-t...