batch upload forks off thumbnail creation in parallel batch upload forks off thumbnail creation in parallel
 

News:

CPG Release 1.6.26
Correct PHP8.2 issues with user and language managers.
Additional fixes for PHP 8.2
Correct PHP8 error with SMF 2.0 bridge.
Correct IPTC supplimental category parsing.
Download and info HERE

Main Menu

batch upload forks off thumbnail creation in parallel

Started by rrauenza, June 24, 2005, 01:50:54 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

rrauenza

I already hit the GD memory limit and increased it -- I found my httpd processes consumed so much memory my system started to swap.  So, I switched to ImageMagick -- and found that during the batch upload that there are a couple of converts for each image uploaded, and the converts are done in parallel.  This appears to be true with httpd, too.  With just three images in a batch upload, my 320MB linux box ran out of memory.  Granted, these aren't small images.

In the previous photo gallery software I used (I'm revaluating after a system crash), the thumbnails were generated on demand (but only once) as they were accessed in the galleries.

Is there a throttle for coppermine to limit how much of the converts it does in parallel?  Or to make it do it in serial?

Versions:

[root@tendo ~]# grep Image rpm.list.txt
ImageMagick-6.2.2.0-2.fc3
[root@tendo ~]# grep http rpm.list.txt
system-config-httpd-1.3.1-1
httpd-suexec-2.0.52-3.1
httpd-2.0.52-3.1
[root@tendo ~]# grep php rpm.list.txt
php-pear-4.3.11-2.5
php-mysql-4.3.11-2.5
php-4.3.11-2.5
php-gd-4.3.11-2.5
[root@tendo ~]# grep gd-2 rpm.list.txt
gd-2.0.28-1.30.1
[root@tendo ~]#

rrauenza


Ah -- Looking at this more -- I see part of the problem.  When you get to the screen of what files to upload, a convert for each img is forked off in parallel there (probably due to parallel http requests for the thumbnails).  If you go ahead and select the images and submit them, then another convert is forked for each image again, and you end up with N*2 converts running in the background.  I haven't tried this with more than three files. 

Maybe a counting semaphore of some type could be used to restrict the actual number of converts happening in parallel?  Or maybe there is one already... I'll start sifting through the code.