[solved] > 1k users --> very slow

[russian_knight]

Hello. My gallery have already above 1k users. From clicking "users" till geting list i wait about 15-20 sec. Is it my or Coppermine bug? What can i do to optimize this?

[omniscientdeveloper]

Set it to debug mode to see how long it takes to generate the page. There can be many factors for slow performance.


-omni

[russian_knight]

USER:
------------------
Array
(
    [ID] => 0ffa5a71d97a39c4bf169a8bfaa10189
    [am] => 1
    [liv] => Array
        (
           

• => 5735
              [1] => 5734
              [2] => 5733
              [3] => 5732
              [4] => 5731
          )
  
      [lap] => 1
      [sort] => da
      [uid] => 1030
      [theme] => mac_ox_x
  )
  
  ==========================
  USER DATA:
  ------------------
  Array
  (
      [user_id] => 1
      [user_group] => 1
      [user_active] => YES
      [user_name] => russian_knight
      [user_password] => ********
      [user_lastvisit] => 2004-06-24 21:17:12
      [user_regdate] => 2004-01-27 13:47:12
      [user_group_list] =>
      [user_email] =>
      [user_website] =>
      [user_location] =>
      [user_interests] =>
      [user_occupation] =>
      [user_actkey] =>
      [disk_max] => 5000
      [disk_min] => 5000
      [can_rate_pictures] => 1
      [can_send_ecards] => 1
      [ufc_max] => 1
      [ufc_min] => 1
      [custom_user_upload] => 0
      [num_file_upload] => 5
      [num_URI_upload] => 3
      [can_post_comments] => 1
      [can_upload_pictures] => 1
      [can_create_albums] => 1
      [has_admin_access] => 1
      [pub_upl_need_approval] => 0
      [priv_upl_need_approval] => 0
      [group_name] => Administrators
      [upload_form_config] => 1
      [group_quota] => 5000
      [can_see_all_albums] => 1
      [group_id] => 1
      [groups] => Array
          (
              [1] => 1
          )
  
  )
  
  ==========================
  Queries:
  ------------------
  Array
  (
     
• => SELECT extension, mime, content FROM cpg11d_filetypes;
      [1] => SELECT * FROM cpg11d_users WHERE user_id='1'AND user_active = 'YES' AND user_password != '' AND BINARY MD5(user_password) = 'cd893489f8e298b91297c2c31c2db094'
      [2] => SELECT MAX(group_quota) as disk_max, MIN(group_quota) as disk_min, MAX(can_rate_pictures) as can_rate_pictures, MAX(can_send_ecards) as can_send_ecards, MAX(upload_form_config) as ufc_max, MIN(upload_form_config) as ufc_min, MAX(custom_user_upload) as custom_user_upload, MAX(num_file_upload) as num_file_upload, MAX(num_URI_upload) as num_URI_upload, MAX(can_post_comments) as can_post_comments, MAX(can_upload_pictures) as can_upload_pictures, MAX(can_create_albums) as can_create_albums, MAX(has_admin_access) as has_admin_access, MIN(pub_upl_need_approval) as pub_upl_need_approval, MIN( priv_upl_need_approval) as  priv_upl_need_approval FROM cpg11d_usergroups WHERE group_id in (1)
      [3] => SELECT group_name FROM  cpg11d_usergroups WHERE group_id= 1
      [4] => DELETE FROM cpg11d_banned WHERE expiry < 1088332961
      [5] => SELECT * FROM cpg11d_banned WHERE ip_addr='10.151.112.25' OR ip_addr='10.151.112.25' OR user_id=1
      [6] => DELETE FROM cpg11d_users WHERE user_name = '' LIMIT 1
      [7] => SELECT count(*) FROM cpg11d_users WHERE 1
      [8] => SELECT user_id, user_name, user_email, UNIX_TIMESTAMP(user_regdate) as user_regdate, UNIX_TIMESTAMP(user_lastvisit) as user_lastvisit, group_name, user_active, COUNT(pid) as pic_count, ROUND(SUM(total_filesize)/1024) as disk_usage, group_quota FROM cpg11d_users AS u INNER JOIN cpg11d_usergroups AS g ON user_group = group_id LEFT JOIN cpg11d_pictures AS p ON p.owner_id = u.user_id GROUP BY user_id ORDER BY user_regdate DESC LIMIT 0, 25
  )
  
  ==========================
  GET :
  ------------------
  Array
  (
  )
  
  ==========================
  POST :
  ------------------
  Array
  (
  )
  
  ==========================
  VERSION INFO :
  ------------------
  PHP version: 4.3.7 - OK
  ------------------
  mySQL version: 3.23.58
  ==========================
  Module: gd
  ------------------
  GD Support enabled
  GD Version 2.0 or higher
  FreeType Support enabled
  FreeType Linkage with freetype
  JPG Support enabled
  PNG Support enabled
  WBMP Support enabled
  ==========================
  Module: mysql
  ------------------
  Active Persistent Links 1
  Active Links 2
  Client API version 3.23.58
  MYSQL_MODULE_TYPE external
  MYSQL_SOCKET /var/lib/mysql/mysql.sock
  MYSQL_INCLUDE -I/usr/include/mysql
  MYSQL_LIBS -L/usr/lib/mysql -lmysqlclient 
  ==========================
  Module: zlib
  ------------------
  ZLib Support enabled
  Compiled Version 1.2.1.1
  Linked Version 1.2.1.1
  ==========================
  Server restrictions (safe mode)?
  ------------------
  Directive | Local Value | Master Value
  safe_mode | Off | Off
  safe_mode_exec_dir | no value | no value
  safe_mode_gid | Off | Off
  safe_mode_include_dir | no value | no value
  safe_mode_exec_dir | no value | no value
  sql.safe_mode | Off | Off
  disable_functions | no value | no value
  file_uploads | On | On
  include_path | .:/usr/share/pear | .:/usr/share/pear
  open_basedir | no value | no value
  ==========================
  email
  ------------------
  Directive | Local Value | Master Value
  sendmail_from | no value | no value
  sendmail_path | /usr/sbin/sendmail -t -i | /usr/sbin/sendmail -t -i
  SMTP | localhost | localhost
  smtp_port | 25 | 25
  ==========================
  Size and Time
  ------------------
  Directive | Local Value | Master Value
  max_execution_time | 30 | 30
  max_input_time | 60 | 60
  upload_max_filesize | 2M | 2M
  post_max_size | 8M | 8M
  ==========================
  Page generated in 25.897 seconds - 9 queries in 25.825 seconds - Album set :

[omniscientdeveloper]

Page generated in 25.897 seconds - 9 queries in 25.825 seconds

Hmm...


[edit]

Post a link with a test user account.

[/edit]

[russian_knight]

Sorry, but gallery is in LAN.

You think it is my problem, not the problem of the way Coppermine using DB?

Someone, who ownes galery with > 1k users, comment please.

P.S. Russian translation is 90% ready.

[Joachim Müller]

this has been asked before: we need to limit the query on the memberlist page, with tabs at the bottom that let you go to the next page. Others who run a gallery with a lot of users reported similar issues in the past.

@russian_knight: unless you can code this yourself, you'll have to wait for some dev team member to have this fixed. Consider it a bug if you want.

@Omni: care to assign this issue to you? Afaik Oasis has coded a workaround for his huge gallery (although cpg1.2.1), as he had run into similar issues in the past.

GauGau

[omniscientdeveloper]

No problem. I think that there should be some free stress testing software out there also.


-omni

[omniscientdeveloper]

Fixed in stable and devel
files: sql/update.sql and usermgr.php (for updates)


You should be able to download these two files from webcvs and run update.php. The problem was in a missing index "owner_id".




-omni

[russian_knight]

Sorry, but:
Page generated in 48.855 seconds - 9 queries in 48.785 seconds - Album set :

I do everything what You say. (download, remove, update.php)

[omniscientdeveloper]

verify in your database with phpmyadmin that the table cpg11d_pictures has an index column "owner_id". I tested it on mine with 4k+ users.


-omni


[edit]

download update.sql again. it was missing a ";"

[/edit]

[GGallery]

execute all these mysql queries 'by hand' and check what's so slow.
also check sort_buffer and key_buffer in /etc/my.cnf

[russian_knight]

download update.sql again. it was missing a ";"

Thanks!!
Page generated in 0.158 seconds - 9 queries in 0.083 seconds - Album set :

[Joachim Müller]

is this solved then?

GauGau

[russian_knight]

Yes, but i`v got another error. I don`t know is it linked with previous... On pushing "My profile" i get:

There was an error while processing a database query.

While executing query "SELECT user_name, user_email, user_group, UNIX_TIMESTAMP(user_regdate) as user_regdate, group_name, user_location, user_interests, user_website, user_occupation, user_group_list, COUNT(pid) as pic_count, ROUND(SUM(total_filesize)/1024) as disk_usage, group_quota FROM cpg11d_users AS u INNER JOIN cpg11d_usergroups AS g ON user_group = group_id LEFT JOIN cpg11d_pictures AS p ON p.owner_id = u.user_id WHERE user_id ='1' GROUP BY user_id " on 0

mySQL error: Unknown column 'user_location' in 'field list'

File: /var/www/html/photo/include/functions.inc.php - Line: 110

[Casper]

Did you download the update.sql from stable, or devel.  The devel one will cause this if used on a stable install, as there are differences.
Check your database, users table, and look for,

'user_location'.  I suspect it does not exist anymore, but you will have 'user_profile1'.

If this is the case, you will need to change the profile fields back, as below;

Code Select Expand

Change `user_profile1` to `user_location`
Change `user_profile2` to `user_interests`
Change `user_profile3` to `user_website`
Change `user_profile4` to `user_occupation`

Note, you can leave profile5 and profile6.  They will be used in version 1.4, and will cause no harm now.

[russian_knight]

Oh, yes, i downloaded develop version. After renaming fields all is OK. Thank you =)

P.S. Is there any chance to add in users table field IP? I need that to kill identical accaunts.

[omniscientdeveloper]

is this solved then?

yea.


-omni

[u3uk]

P.S. Russian translation is 90% ready.

100%