Two web servers + One DB Server - Optimization Needed

  • Hi guys,

    We have three servers running vBulletin and fine tuning is needed right now. Below I am posting all required information you asked, below I will post another message with additional information and some thoughts.

    Required info:

    1. Is this on dedicated or shared virual server. If shared, how many sites share this server (ask web host if needed)

    Three dedicated servers, two webservers and one DB server

    2. Your server specs.

    server 1: web server with Apache 2.0.59 + PHP 5.2.1
    server 2: web server with Apache 2.0.59 + PHP 5.2.1
    server 3: DB server with MySQL 5.0.37

    Servers 1 and 2 are dual-Xeon 3 GHz (with HT enabled) with 2 GB RAM and 80 GB HDD EIDE non RAID, load balance with Pound.

    Server 3 is identical but with 4 GB RAM.

    Servers running Debian 3.1 kernel 2.6.8-3-686-smp

    3. what version of vB are you running ?

    3.6.7 PL1

    4. if you use mysql 4.x instead of mysql 3.23.x, do you have any innodb type databases/tables on your server ?

    No, MyISAM.

    5. if possible how mysql was compiled/installed

    Binaries compiled with Intel compiler downloaded directly from mysql.com

    6. your top stats

    Max users online: 2,754 yesterday
    435,800 threads
    2,230,000 messages
    157,850 members

    7. your mysql configuration variables located at /etc/my.cnf or c:my.cnf or my.ini so post the contents inside of my.cnf (minus any passwords of course).

    Here is everything that isn't commented:

    [mysqld]
    port = 3306
    socket = /tmp/mysql.sock
    safe-show-database
    skip-innodb
    max_connections = 800
    key_buffer = 512M
    myisam_sort_buffer_size = 128M
    join_buffer_size = 2M
    read_buffer_size = 2M
    sort_buffer_size = 8M
    table_cache = 5000
    thread_cache_size = 768
    wait_timeout = 14400
    connect_timeout = 10
    tmp_table_size = 512M
    max_heap_table_size = 512M
    max_allowed_packet = 128M
    max_connect_errors = 10
    read_rnd_buffer_size = 8M
    bulk_insert_buffer_size = 16M
    query_cache_limit = 3M
    query_cache_size = 32M
    query_cache_type = 1
    query_prealloc_size = 163840
    query_alloc_block_size = 32768

    [mysqld_safe]
    nice = -10
    open_files_limit=8192
    [mysqldump]
    quick
    max_allowed_packet = 16M
    [mysql]
    no-auto-rehash
    [isamchk]
    key_buffer = 20M
    sort_buffer_size = 20M
    read_buffer = 2M
    write_buffer = 2M
    [myisamchk]
    key_buffer = 64M
    sort_buffer = 64M
    read_buffer = 16M
    write_buffer = 16M
    [mysqlhotcopy]
    interactive-timeout

    8. MySQL stats from ssh telnet as root user type:

    status.txt and mysqlreport.txt so this message won't be too long.

    9. is your vB the only thing on the server? or other scripts & sites which utilise php and mysql?

    All servers are dedicated to vBulletin only. On server 3 we also have memcached running.

    10. how many average and max concurrent users on your vB forum ? and what your cookie timeout is ?

    Don't know.

    11. create a file named phpinfo.php and place this code in it and post the url/link to it from your web site

    http://forum.clubedohardware.com.br/phpinfo.php

    12. if you run Apache and you have your own dedicated server or access to your httpd.conf (apache configuration file) can you post the values you have set for the following :

    KeepAlive On
    MaxKeepAliveRequests [didn't find this option]
    KeepAliveTimeout 5
    MinSpareServers 5
    MaxSpareServers 32
    StartServers 5
    MaxClients 256
    Maxrequestsperchild value = 64

    13. check to see if any files i.e. apache log files are hitting 2GB or 4GB max file size limits i.e. see if you have max file size exceeded messages in apache error log

    No

    14. Post output from these 3 commands

    Got the same results for the three servers. See below.

    uname -a

    Debian 2.6.8-3-686-smp

    ulimit -aH

    core file size (blocks, -c) unlimited
    data seg size (kbytes, -d) unlimited
    file size (blocks, -f) unlimited
    pending signals (-i) unlimited
    max locked memory (kbytes, -l) unlimited
    max memory size (kbytes, -m) unlimited
    open files (-n) 1024
    pipe size (512 bytes, -p) 8
    POSIX message queues (bytes, -q) unlimited
    stack size (kbytes, -s) unlimited
    cpu time (seconds, -t) unlimited
    max user processes (-u) unlimited
    virtual memory (kbytes, -v) unlimited
    file locks (-x) unlimited

    cat /proc/cpuinfo

    in attach in order to not make this thread very long.

    This ends the required info you asked. I will post a new message below with some additional info and thoughts.

    Thanks,
    Gabriel.


  • Here are some additional info and thoughts.

    I think our MySQL server (server 3 above) is fine. Load is low (load average: 0.40, 0.36, 0.33 right now). We have memcached installed on this server.

    Current memcached stats are:

    Host xx.xx.xx.xx:11211
    Latency Test 0.16 ms
    MB Allocated 512.00
    MB Used 0.04
    Current Items 25
    Total Items 1,770
    PID 11661
    Daemon Uptime 0d, 21h, 21m, 1s
    Server Time May 25, 2007, 6:17:06
    Version 1.2.0
    Pointer Size 32
    RUsage User 55.790518
    RUsage System 128.579452
    Current Connections 2
    Total Connections 410,972
    Connection Structures 35
    Gets 5,876,153
    Sets 1,770
    Get Hits 5,876,128
    Get Misses 25
    MB Read 92.11
    MB Written 15,939.54

    The problem, I think, are the webservers. As mentioned above, we are using Pound to make the load balance between them. It is working fine, except that I think that load is a little bit high. Right now this is what we have:

    server 1: load average: 3.46, 4.23, 3.58
    server 2: load average: 1.87, 2.19, 2.12

    Probably server 1 load is higher because pound is running there (it listens there on port 80 and then Apache listens on port 8080 on both servers).

    Maybe we also need php optimization. We have Xcache installed. Here are the main php.ini variables that are not commented out:

    max_execution_time = 30 ; Maximum execution time of each script, in seconds
    max_input_time = 60 ; Maximum amount of time each script may spend parsing request data
    memory_limit = 128M ; Maximum amount of memory a script may consume (128MB)
    post_max_size = 8M
    default_socket_timeout = 60
    ; Memcache
    extension=/usr/local/apache2/php/lib/php/extensions/no-debug-non-zts-20060613/memcache.so
    memcache.allow_failover=0
    [xcache-common]
    zend_extension = /usr/local/apache2/php/lib/php/extensions/no-debug-non-zts-20060613/xcache.so
    ; required for >=php5.1 if you turn XCache on
    auto_globals_jit = Off
    [xcache.admin]
    xcache.admin.user = .admin.
    ; paste the output of md5(.password.) below
    xcache.admin.pass = ..
    [xcache]
    xcache.size = 128M
    xcache.count = 4
    xcache.slots = 8K
    xcache.mmap_path = ./dev/zero.
    xcache.cacher = On
    xcache.optimizer = On
    mysql.allow_persistent = On
    mysql.connect_timeout = 60
    mysqli.max_links = -1
    session.cookie_lifetime = 0
    session.gc_probability = 1
    session.gc_divisor = 100
    session.gc_maxlifetime = 1440
    session.cache_expire = 180

    I expected to see more options at php.ini, but there is no more than this.

    Please advice. Suggestions are welcome.

    Thanks,
    Gabriel.


  • Thanks for the configuration tips. I had already installed Xcache. Thanks for the dual NIC tip, I am looking into that.

    I was also wondering if there is any adjustments I should do at php.ini. Any comments about my php.ini?

    Thanks,
    Gabriel.


  • if all is well now... then your current php.ini should be fine :)


  • Try the following in this exact order. You can ignore any of the suggestions that you have already done.

    1. Upgrade PHP to 5.2.2 or if you have problems install PHP 4.4.6 (in either case install as ISAPI module and NO CGI)
    2. Edit /etc/my.cnf and place the following mysql server settings in /etc/my.cnf and restart mysql server afterwards


    [mysqld]
    port = 3306
    socket = /tmp/mysql.sock
    back_log = 150
    skip-innodb
    max_connections = 800
    key_buffer = 512M
    myisam_sort_buffer_size = 64M
    join_buffer_size = 2M
    read_buffer_size = 2M
    sort_buffer_size = 4M
    table_cache = 1800
    thread_cache_size = 512
    wait_timeout = 35
    connect_timeout = 10
    tmp_table_size = 256M
    max_heap_table_size = 256M
    max_allowed_packet = 64M
    max_connect_errors = 10
    thread_concurrency = 4
    concurrent_insert = 2
    table_lock_wait_timeout = 35
    read_rnd_buffer_size = 1M
    bulk_insert_buffer_size = 8M
    query_cache_limit = 6M
    query_cache_size = 96M
    query_cache_type = 1
    query_prealloc_size = 131072
    query_alloc_block_size = 65536
    default-storage-engine = MyISAM

    [mysqld_safe]
    nice = -10
    open_files_limit = 8192

    [mysqldump]
    quick
    max_allowed_packet = 16M

    [myisamchk]
    key_buffer = 64M
    sort_buffer = 64M
    read_buffer = 16M
    write_buffer = 16M


    3. Install APC Cache lastest version from http://pecl.php.net/package/APC. Read install guide at http://www.vbulletin.com/forum/showthread.php?t=165367 OR try the alternative PHP caching software, Xcache which seems to be a bit faster than APC Cache - http://www.vbulletin.com/forum/showthread.php?t=213267

    4. If you just upgraded to vB 3.5.x/3.6.x try to disable these 4 options:

    Admin CP -> vBulletin Options -> Forums Home Page Options -> Display Logged in Users?

    Admin CP -> vBulletin Options -> Forum Display Options (forumdisplay) -> Show Users Browsing Forums

    Admin CP -> vBulletin Options -> Thread Display Options -> Show Users Browsing Thread

    Admin CP -> vBulletin Options -> Message Searching Options -> Automatic Similar Thread search

    5. Edit httpd.conf values to following and restart apache

    KeepAlive: On
    MaxKeepAliveRequests: 100
    KeepAliveTimeout: 4
    MinSpareServers: 15
    MaxSpareServers: 25
    StartServers: 15
    MaxClients: 200
    MaxRequestsPerChild: 1000

    6. If you have split web + db servers, ensure EACH web server has dual network cards as outlined at http://www.vbulletin.org/forum/showthread.php?t=111191







  • #If you have any other info about this subject , Please add it free.#
    Your name:
    E-mail:
    Telphone:

    Your comments:


    If you have any other info about Two web servers + One DB Server - Optimization Needed , Please add it free.