Extremely high CPU load

[monstermind]

Hi guys,

Since I installed CPG145, the CPU load on my dual xeon 2.8 ghz, 2gb ram webserver went from an average 5-10% to a 40-50% CPU load and beyond.

Is this CPU load normal?
What is causing this CPU load?
Is there a solution?

Thanks in advance and regards,

Robert-Jan

[monstermind]

My CPG website has 2,100 unique visitors a day who produce about 50,000 pageviews. The website transfers about 18 GB a day in traffic.

I use the "Modded by Stramm" modpack and ALL statistics and logging is disabled.

The same webserver also runs my other websites, which use php/mysql scripting as well. Those websites get about 85,000 unique visitors a day. And that only gives 5-10% CPU load. This "little" 2,100-uniques-a-day CPG site caused may load to multiply by 5.

When giving the "top" command on the UNIX root, I see that the load is caused by apache2. Sometimes I see 3-4 apache2 processes running simultaniously, who demand 12% CPU load each.

Thanks in advance and regards,

Robert-Jan

[Joachim Müller]

link?

[monstermind]

(Adult website) www.ishotmyself.nl

[Stramm]

Continuously? That processes with 10+% CPU may occur if a user uploads files (depending on your OS, overall load even more)

[monstermind]

Yes, I am continiously having 50%+ CPU load. Even when nobody is uploading pictures, I still have several apache processes with 12-20% CPU load each. (The "top" command refreshes, sometimes it's just one, next refresh might be 5 processes).

I even see peaks of 75% CPU load...

[Stramm]

enable debug for admin to find out what queries need the most time/ cpu

[monstermind]

Since I cannot post long messages on this board (hitting "post" button causes script to hang), here's the debug output: http://www.ishotmyself.nl/error.html

[Stramm]

disable displaying of comments count below the thumbs, each count needs a query -> 15 thumbs per page, 15 extra queries

I guess you've private albums enabled?? That may suck a lot of CPU as well

You need a lot of queries displaying a thumb page. Reduce that...
compare to my site. I've enabled debug
http://stramm.st.funpic.org/thumbnails.php?album=8

that's 35 queries. If I disable displaying comments count -> 23 queries.
I have additional user info enabled (pm and buddy list info) -> disabling it -4 queries
I have the plugin system enabled -> disabling it -1 query for the system and -2 queries for the online plugin
If you do not use login / and or banning system you can disable it (modifying code) -> -2 queries

If I'd optimize max I'd have 14 queries (it's not the same for each install, also depends on the number of albums/ cats etc) compared to your 62

But that seems to be a nice box 

Page generated in 0.399 seconds - 62 queries in 0.005 seconds

[monstermind]

OK, I'm going to try reducing queries...

Those other messages, about undefined variables, does that need work?

Thanks a lot so far!

Regards, Robert-Jan

P.S. Are you (or any of the other wizards) available for paid customization of CPG?

[monstermind]

I already had private albums disabled.

Thumbnail page original config:
Page generated in 0.389 seconds - 43 queries in 0.004 seconds

Disabled "x comments" and "viewed x times" below thumbs:
Page generated in 0.42 seconds - 18 queries in 0.008 seconds

Reduced # of queries by 60%, hopefully this helps

[monstermind]

BTW, homepage gives 42 queries, intermediate pic 22 queries. Any room for optimization there?

I'm still curious if you (or one of the other wizards) is available for paid custimization of CPG...

[Abbas Ali]

I'm still curious if you (or one of the other wizards) is available for paid custimization of CPG...

Start  a new thread in Looking for Freelancers / Paid help board. Do read sub-board rules, read first!.

[monstermind]

I've reduced the number of queries, but I have ZERO results. CPU load is still 40-50% and up... And I still have multiple apache 2 processes running at the same time, demanding 10-20% CPU load each.

Any ideas where to look?

Thanks and regards,

Robert-Jan

[Joachim Müller]

get rid of the last additions meta album or reduce the number of rows if that's an option for you.

[trippinsweet]

You could probably try installing a php cache like eAccellerator or turning on query caching in mysql, while disabling ratings and hit counting for cpg.

I used to have server loads as high as you, but doing both of those things brought the beast under control.

[blackmamba]

Hi,
I'm getting 80%-95% CPU load with Coppermine Gallery 1.4.8 (the same with lower versions) and slow page load:
Page generated in 4.881 seconds - 27 queries in 0.127 seconds - Album set : ; Meta set:
My web server gets a load of ~2000visits/day and ~60.000 pages/day
CPG gallery here

Hardware stuff:
DELL - Intel PIII 933 Mhz, 500 RAM, HDD maxtor 80Gb, 3COM networking, optical fiber broadband WAN.

Soft:
WinXP SP2, latest XAMPP (apache, php & mysql)

Tuning server:
WinXP: disabled every services not needed (indexing services, restore, wireless ... etc)
apache - memcache on, info&status off, deflate on, hostlookup off ..etc
mysql - cache query, cache threads .. etc (myhuge.ini)
php - zend accelerator level 15

As I've seen, the apache & mysql works great with minimum CPU load when serving static html pages (less then 0.003sec even on a high load)
The problem seems to be on the PHP(cpg) side

Any other clue?


debug info:

Code Select Expand


USER:
------------------
Array
(
    [ID] => a450e721ec91c6d6831cde0d1e5f48e4
    [am] => 1
    [lang] => romanian
    [liv] => Array
        (
            [0] => 2
            [1] => 2665
            [2] => 2368
            [3] => 2367
            [4] => 2343
        )

    [search] => Array
        (
            [search] => elena udrea
            [album] => search
            [title] => on
            [newer_than] =>
            [caption] => on
            [older_than] =>
            [keywords] => on
            [type] => OR
            [filename] => on
            [params] => Array
                (
                    [title] => on
                    [caption] => on
                    [keywords] => on
                    [filename] => on
                )

        )

    [sort] => dd
)

==========================
USER DATA:
------------------
Array
(
    [user_id] => 1
    [user_name] => Admin
    [groups] => Array
        (
            [0] => 1
        )

    [disk_max] => 0
    [disk_min] => 0
    [can_rate_pictures] => 1
    [can_send_ecards] => 1
    [ufc_max] => 3
    [ufc_min] => 3
    [custom_user_upload] => 0
    [num_file_upload] => 10
    [num_URI_upload] => 10
    [can_post_comments] => 1
    [can_upload_pictures] => 1
    [can_create_albums] => 1
    [has_admin_access] => 1
    [pub_upl_need_approval] => 0
    [priv_upl_need_approval] => 0
    [group_name] => Administrators
    [upload_form_config] => 3
    [group_quota] => 0
    [can_see_all_albums] => 1
    [group_id] => 1
)

==========================
Queries:
------------------
Array
(
    [0] => SELECT extension, mime, content, player FROM cpg143_filetypes; (0.001s)
    [1] => delete from `pozehaioase`.cpg143_sessions where time<1151664931 and remember=0; (0.002s)
    [2] => delete from `pozehaioase`.cpg143_sessions where time<1150458931; (0.001s)
    [3] => select user_id from `pozehaioase`.cpg143_sessions where session_id=md5("82f9c02dbacb4d06881fb1e9286070734afd313dddb556d47a12be66200bc3ae"); (0.001s)
    [4] => select user_id as id, user_password as password from `pozehaioase`.cpg143_users where user_id=1 (0s)
    [5] => SELECT u.user_id AS id, u.user_name AS username, u.user_password AS password, u.user_group+100 AS group_id FROM `pozehaioase`.cpg143_users AS u INNER JOIN `pozehaioase`.cpg143_usergroups AS g ON u.user_group=g.group_id WHERE u.user_id='1' (0s)
    [6] => SELECT user_group_list FROM `pozehaioase`.cpg143_users AS u WHERE user_id='1' and user_group_list <> ''; (0s)
    [7] => SELECT MAX(group_quota) as disk_max, MIN(group_quota) as disk_min, MAX(can_rate_pictures) as can_rate_pictures, MAX(can_send_ecards) as can_send_ecards, MAX(upload_form_config) as ufc_max, MIN(upload_form_config) as ufc_min, MAX(custom_user_upload) as custom_user_upload, MAX(num_file_upload) as num_file_upload, MAX(num_URI_upload) as num_URI_upload, MAX(can_post_comments) as can_post_comments, MAX(can_upload_pictures) as can_upload_pictures, MAX(can_create_albums) as can_create_albums, MAX(has_admin_access) as has_admin_access, MIN(pub_upl_need_approval) as pub_upl_need_approval, MIN( priv_upl_need_approval) as  priv_upl_need_approval FROM cpg143_usergroups WHERE group_id in (1) (0.001s)
    [8] => SELECT group_name FROM  cpg143_usergroups WHERE group_id= 1 (0.001s)
    [9] => update `pozehaioase`.cpg143_sessions set time='1151668531' where session_id=md5('82f9c02dbacb4d06881fb1e9286070734afd313dddb556d47a12be66200bc3ae'); (0.001s)
    [10] => SELECT user_favpics FROM cpg143_favpics WHERE user_id = 1 (0.001s)
    [11] => DELETE FROM cpg143_banned WHERE expiry < '2006-06-30 13:55:32' (0.001s)
    [12] => SELECT * FROM cpg143_banned WHERE (ip_addr='85.120.249.141' OR ip_addr='85.120.249.141' OR user_id=1) AND brute_force=0 (0s)
    [13] => SELECT cid, name, description, thumb FROM cpg143_categories WHERE parent = ''  ORDER BY pos (0.001s)
    [14] => SELECT aid FROM cpg143_albums as a WHERE category>=10000 (0s)
    [15] => SELECT count(*) FROM cpg143_pictures as p, cpg143_albums as a WHERE p.aid = a.aid AND approved='YES' AND category >= 10000 (0.001s)
    [16] => SELECT aid FROM cpg143_albums as a WHERE category = '0' (0.001s)
    [17] => SELECT count(*) FROM cpg143_albums as a WHERE 1 (0s)
    [18] => SELECT count(*) FROM cpg143_pictures as p LEFT JOIN cpg143_albums as a ON a.aid=p.aid WHERE 1 AND approved='YES' (0.007s)
    [19] => SELECT count(*) FROM cpg143_comments as c LEFT JOIN cpg143_pictures as p ON c.pid=p.pid LEFT JOIN cpg143_albums as a ON a.aid=p.aid WHERE 1 (0.035s)
    [20] => SELECT count(*) FROM cpg143_categories WHERE 1 (0s)
    [21] => SELECT sum(hits) FROM cpg143_pictures as p LEFT JOIN cpg143_albums as a ON p.aid=a.aid WHERE 1 (0.017s)
    [22] => SELECT COUNT(*) FROM cpg143_pictures WHERE approved = 'NO' (0.005s)
    [23] => SELECT count(*) FROM cpg143_albums as a WHERE category = '0' (0.001s)
    [24] => SELECT a.aid, a.title, a.description, category, visibility, filepath, filename, url_prefix, pwidth, pheight FROM cpg143_albums as a LEFT JOIN cpg143_pictures as p ON a.thumb=p.pid WHERE category=0 ORDER BY a.pos LIMIT 0,12 (0.003s)
    [25] => SELECT a.aid, count( p.pid )  AS pic_count, max( p.pid )  AS last_pid, max( p.ctime )  AS last_upload, a.keyword FROM cpg143_albums AS a  LEFT JOIN cpg143_pictures AS p ON a.aid = p.aid AND p.approved =  'YES' WHERE a.aid IN (5, 8, 2, 12, 7, 1, 4, 10, 11, 3, 6, 9)GROUP BY a.aid (0.045s)
    [26] => SELECT filepath, filename, url_prefix, pwidth, pheight FROM cpg143_pictures WHERE pid='2633' (0.001s)
)

==========================
GET :
------------------
Array
(
)

==========================
POST :
------------------
Array
(
)

==========================
VERSION INFO :
------------------
PHP version: 5.1.4 - OK
------------------
mySQL version: 5.0.21-community-nt
------------------
Coppermine version: 1.4.6(stable)
==========================
Module: GD
------------------
GD Version: bundled (2.0.28 compatible)
FreeType Support: 1
FreeType Linkage: with freetype
T1Lib Support: 1
GIF Read Support: 1
GIF Create Support: 1
JPG Support: 1
PNG Support: 1
WBMP Support: 1
XPM Support:
XBM Support: 1
JIS-mapped Japanese Font Support:

==========================
Module: mysql
------------------
MySQL Supportenabled
Active Persistent Links 0
Active Links 1
Client API version 5.0.21
==========================
Module: zlib
------------------
ZLib Support enabled
Stream Wrapper support compress.zlib://
Stream Filter support zlib.inflate, zlib.deflate
Compiled Version 1.2.3
Linked Version 1.2.3
==========================
Server restrictions (safe mode)?
------------------
Directive | Local Value | Master Value
safe_mode | Off | Off
safe_mode_exec_dir | no value | no value
safe_mode_gid | Off | Off
safe_mode_include_dir | no value | no value
safe_mode_exec_dir | no value | no value
sql.safe_mode | Off | Off
disable_functions | no value | no value
file_uploads | On | On
include_path | .;d:\xampp\xampp\php\pear\ | .;d:\xampp\xampp\php\pear\
open_basedir | no value | no value
==========================
email
------------------
Directive | Local Value | Master Value
sendmail_from | no value | no value
sendmail_path | no value | no value
SMTP | localhost | localhost
smtp_port | 25 | 25
==========================
Size and Time
------------------
Directive | Local Value | Master Value
max_execution_time | 60 | 60
max_input_time | 60 | 60
upload_max_filesize | 16M | 16M
post_max_size | 16M | 16M
==========================
Page generated in 4.881 seconds - 27 queries in 0.127 seconds - Album set : ; Meta set: ;


thanks in advance.

[Nibbler]

Are you using lang fallback? If so try disabling it.

[blackmamba]

I use the
- default english language file (modified by myself for the romanian visitors),
- no fallback for language
- no language flags
- no change theme option for the users
- no language selection option for the users
- the classic theme
- no extra scripts (except users online - a small one)
- I've even deleted the sort thumbnail option in album view,
- Plugins disabled
- No pic rating for the unregistered visitors
- BB code disabled
- no simles in comments
- no loging
- no stats
- no categories, only 12 albums
- all the pics are progressive jpeg enhanced

... so, it's a spartan configuration still high CPU load 85-95% with only 50-maximum 70 users online

[Sami]

1. provide a link to your 1.4.8 gallery (with debug for everyone enable)
2. upgrade your 1.4.6
3. please do not remove copyright