Batch Upload - Locking Php

[richarn]

I have been working on getting my batch upload working and have run into a bit of a problem.  I am using GD 2.x as image converter.  I am an ex windows software dev turned Professional Photographer.  My files are rather large, however, it seems to even have the same issue on smaller files it just takes longer.  This leads me to believe that it has something to do with memory usage.  What occurs is during the SearchNew.php search for new files in the directory it starts showing images that it has resized then at a certain point they become broken links.  The point at which the link becomes broken depends on the sizes of the files it has already processed.  Now the thing that I am having the real issue with is at this point it has hosed the Php process.  If I click on the broken link I get "Service Unavailable".  If I click any other links in Coppermine I get "Service Unavailable".  The only way to clear it is to Navigate out of anything related to php or to cycle the php service on the server.  I realize this is probably something my hosting provider has me configured for but what I need to find out is what it might be.  That way I can figure out if they can remedy this or maybe there is a way in my Coppermine install to fix this.   Below is Debug info from SearchNew.php.  Note... I probably won't have any say in changes on the php configuration since I am on a shared server.  Thanks in advance for any help anyone can give me for this.

********************** Debug Info ********************
USER:
------------------
Array
(
    [ID] => 4038d313aee914fe0ec1b03df8b73acb
    [am] => 1
    [liv] => Array
        (
           

• => 42
              [1] => 35
              [2] => 27
              [3] => 25
              [4] => 14
          )
  
  )
  
  ==========================
  USER DATA:
  ------------------
  Array
  (
      [user_id] => 1
      [user_group] => 1
      [user_active] => YES
      [user_name] => richarn
      [user_password] => ********
      [user_lastvisit] => 2005-06-29 18:56:59
      [user_regdate] => 2005-06-29 18:52:55
      [user_group_list] =>
      [user_email] =>
      [user_website] =>
      [user_location] =>
      [user_interests] =>
      [user_occupation] =>
      [user_actkey] =>
      [disk_max] => 0
      [disk_min] => 0
      [can_rate_pictures] => 1
      [can_send_ecards] => 1
      [ufc_max] => 3
      [ufc_min] => 3
      [custom_user_upload] => 0
      [num_file_upload] => 5
      [num_URI_upload] => 3
      [can_post_comments] => 1
      [can_upload_pictures] => 1
      [can_create_albums] => 1
      [has_admin_access] => 1
      [pub_upl_need_approval] => 0
      [priv_upl_need_approval] => 0
      [group_name] => Administrators
      [upload_form_config] => 3
      [group_quota] => 0
      [can_see_all_albums] => 1
      [group_id] => 1
      [groups] => Array
          (
              [1] => 1
          )
  
  )
  
  ==========================
  Queries:
  ------------------
  Array
  (
     
• => SELECT extension, mime, content FROM cpg133_filetypes;
      [1] => SELECT * FROM cpg133_users WHERE user_id='1'AND user_active = 'YES' AND user_password != '' AND BINARY MD5(user_password) = '9235867f49ccc59c2cf3cfbaded5af24'
      [2] => SELECT MAX(group_quota) as disk_max, MIN(group_quota) as disk_min, MAX(can_rate_pictures) as can_rate_pictures, MAX(can_send_ecards) as can_send_ecards, MAX(upload_form_config) as ufc_max, MIN(upload_form_config) as ufc_min, MAX(custom_user_upload) as custom_user_upload, MAX(num_file_upload) as num_file_upload, MAX(num_URI_upload) as num_URI_upload, MAX(can_post_comments) as can_post_comments, MAX(can_upload_pictures) as can_upload_pictures, MAX(can_create_albums) as can_create_albums, MAX(has_admin_access) as has_admin_access, MIN(pub_upl_need_approval) as pub_upl_need_approval, MIN( priv_upl_need_approval) as  priv_upl_need_approval FROM cpg133_usergroups WHERE group_id in (1)
      [3] => SELECT group_name FROM  cpg133_usergroups WHERE group_id= 1
      [4] => DELETE FROM cpg133_banned WHERE expiry < '2005-07-16 13:44:10'
      [5] => SELECT * FROM cpg133_banned WHERE ip_addr='67.65.132.119' OR ip_addr='67.65.132.119' OR user_id=1
      [6] => SELECT aid, title FROM cpg133_albums WHERE 1
      [7] => SELECT filepath, filename FROM cpg133_pictures WHERE filepath LIKE 'Upload Area/Cate%'
      [8] => SELECT aid, title FROM cpg133_albums WHERE category = 0 ORDER BY title
      [9] => SELECT DISTINCT a.aid as aid, a.title as title, c.name as cname FROM cpg133_albums as a, cpg133_categories as c WHERE a.category = c.cid AND a.category < '10000' ORDER BY cname,title
      [10] => SELECT aid, CONCAT('(', user_name, ') ', title) AS title FROM cpg133_albums AS a INNER JOIN cpg133_users AS u ON category = (10000 + user_id) ORDER BY title
  )
  
  ==========================
  GET :
  ------------------
  Array
  (
      [startdir] => Upload Area/Cate
  )
  
  ==========================
  POST :
  ------------------
  Array
  (
  )
  
  ==========================
  VERSION INFO :
  ------------------
  PHP version: 4.3.10 - OK
  ------------------
  mySQL version: 4.1.9-standard
  ------------------
  Coppermine version: 1.3.3
  ==========================
  Module: gd
  ------------------
  GD Support enabled
  GD Version bundled (2.0.28 compatible)
  FreeType Support enabled
  FreeType Linkage with freetype
  GIF Read Support enabled
  GIF Create Support enabled
  JPG Support enabled
  PNG Support enabled
  WBMP Support enabled
  XBM Support enabled
  ==========================
  Module: mysql
  ------------------
  Active Persistent Links 0
  Active Links 1
  Client API version 3.23.49
  ==========================
  Module: zlib
  ------------------
  ZLib Support enabled
  Compiled Version 1.1.4
  Linked Version 1.1.4
  ==========================
  Server restrictions (safe mode)?
  ------------------
  Directive | Local Value | Master Value
  safe_mode | Off | Off
  safe_mode_exec_dir | no value | no value
  safe_mode_gid | Off | Off
  safe_mode_include_dir | no value | no value
  safe_mode_exec_dir | no value | no value
  sql.safe_mode | Off | Off
  disable_functions | no value | no value
  file_uploads | On | On
  include_path | .;c:\php4\pear | .;c:\php4\pear
  open_basedir | no value | no value
  ==========================
  email
  ------------------
  Directive | Local Value | Master Value
  sendmail_from | me@localhost.com | me@localhost.com
  sendmail_path | no value | no value
  SMTP | mailhub.registeredsite.com | mailhub.registeredsite.com
  smtp_port | 25 | 25
  ==========================
  Size and Time
  ------------------
  Directive | Local Value | Master Value
  max_execution_time | 30 | 30
  max_input_time | -1 | -1
  upload_max_filesize | 2M | 2M
  post_max_size | 8M | 8M
  ==========================
  Page generated in 0.226 seconds - 11 queries in 0.025 seconds - Album set :

[richarn]

This is still an issue.  It seems to me that what is breaking is the GD image stuff but is happening because the processes in Coppermine are likely not releasing the memory or the GD binaries or library after each use.  That is why it works for a while then breaks and how many files it can process is related to how large the files are.  Please someone help me wth this.

[Joachim Müller]

does indeed sound like improper server setup. Suggested workaround: add less files in one go. If the thumbs get broken, just try refreshing the page by pressing [F5] - the pics the script has processed already should turn up as "DP", while the ones that showed as broken thumbnails should be processed. If your server freezes and needs to be restarted, than that's clearly something your webhost has to figure out. The PHP commands that trigger the GD stuff don't need to be terminated, so there is no other mechanism you could look into (i.e. modify coppermine codes).

[richarn]

For one thing I really cannot add less files at a time or it would make job 10 times longer as I have Galleries of Proofs for my clients that are anywhere from 250-several thousand.  This is giving me trouble after say 10-12 of my regular 800x600 images.  I also cannot keep hitting F5 since in my initial post I stated that it locks the PHP process.  This means every time I navigate or refresh on a PHP page I get a Service Unavailable.  The only way to clear this is to navigate far enough out of the PHP software(to like Yahoo or something like that) or to Cycle the PHP service on the server.  The reason I say it is a memory release issue is that is sort of how it is presenting itself.  If I make the images 640x480 it may process 15-16 before having the same behavior.  If I up it to 1280x1024 Its around 5 of them.  If after each thumb creation from the original file the original image file was released and its memory deallocated I should never hit my memory barrier unless I had one massive file that went over it.  I have seen other posts that have you increase the memory for use in this but that is not possible when being hosted by a large shared hosting service and its only prolonging the inevitable when the limit is reached.  Here is to hoping there may be another answer because I really like Coppermine and would hate to have to change to something else.

[Joachim Müller]

the initial problem is the server setup: if a php script runs into memory outage, the php process mustn't stop and need recycling. As suggested before: contact your webhost for support.

[artistsinhawaii]

Richarn,

It does sound more like a server issue to me as well.  I've batch added 45, 1200x800 pixel, pics at a time without problems on my webhost at Ipower.com.  I've since lowered my res to 1000 x 665 because 1200 just seemed a bit unnecessary.  If your webhost can't get it right, you may want to look around for a different hosting service.


Dennis



 

[richarn]

Yes I know it is some server issue since I too have had it work for me in the past on a different hosting service.  If I had to guess they are limiting the memory or threads on one of the services.  I was hoping there was some way to make sure it didn't happen from the software... Oh well.  Now can anyone point me towards a couple of things they might have configured wrong so I can stear thier tech support?  I am not privy to most of thier settings and I know I am going to get some moronic tech support person that does nothing but look up FAQs and says straight up they dont support 3rd party software.  Thanks for the responses.

Rick

[kegobeer]

Don't bother.  In my experience, webhosts won't respond to any suggestions you may make.  It's their server, they'll run it how they want to.  Just find a better host; it's much easier and you won't be nearly as frustrated.

[richarn]

Yes thats why I was trying other stuff Kego before having to talk to them.  I have already had one run in with the folks at Interland over what they kept telling me was due to a third party software.  It was actually my own code and it wasnt working properly due to thier server setup which I finally proved to them.  Also opened a ticket with them for something that was hanging when you executed it and they closed it with a link to instructions on how to do what I was already doing.  Uggg I just hate to have to move my site yet again as I have some 10,000 pictures on it. 

BTW what is this XP_Publisher thing refered to in some posts?  Is it something in Windows XP?  Is it possible that it can get around my apparent Web Hosting configurations issues? (Nevermind - getting farther into searches I found the explaination and I wont run Windows XP)

Thanks all for the Help

[richarn]

Thanks for all the help - I moved my site to Hostgo who has predefined scripts I can run from thier admin tool to install Coppermine.  Everything installed wonderfully and I was able to process all my pictures without any troubles.