batch upload forks off thumbnail creation in parallel batch upload forks off thumbnail creation in parallel
 

News:

cpg1.5.48 Security release - upgrade mandatory!
The Coppermine development team is releasing a security update for Coppermine in order to counter a recently discovered vulnerability. It is important that all users who run version cpg1.5.46 or older update to this latest version as soon as possible.
[more]

Main Menu

batch upload forks off thumbnail creation in parallel

Started by rrauenza, June 24, 2005, 01:50:54 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

rrauenza

I already hit the GD memory limit and increased it -- I found my httpd processes consumed so much memory my system started to swap.  So, I switched to ImageMagick -- and found that during the batch upload that there are a couple of converts for each image uploaded, and the converts are done in parallel.  This appears to be true with httpd, too.  With just three images in a batch upload, my 320MB linux box ran out of memory.  Granted, these aren't small images.

In the previous photo gallery software I used (I'm revaluating after a system crash), the thumbnails were generated on demand (but only once) as they were accessed in the galleries.

Is there a throttle for coppermine to limit how much of the converts it does in parallel?  Or to make it do it in serial?

Versions:

[root@tendo ~]# grep Image rpm.list.txt
ImageMagick-6.2.2.0-2.fc3
[root@tendo ~]# grep http rpm.list.txt
system-config-httpd-1.3.1-1
httpd-suexec-2.0.52-3.1
httpd-2.0.52-3.1
[root@tendo ~]# grep php rpm.list.txt
php-pear-4.3.11-2.5
php-mysql-4.3.11-2.5
php-4.3.11-2.5
php-gd-4.3.11-2.5
[root@tendo ~]# grep gd-2 rpm.list.txt
gd-2.0.28-1.30.1
[root@tendo ~]#

rrauenza


Ah -- Looking at this more -- I see part of the problem.  When you get to the screen of what files to upload, a convert for each img is forked off in parallel there (probably due to parallel http requests for the thumbnails).  If you go ahead and select the images and submit them, then another convert is forked for each image again, and you end up with N*2 converts running in the background.  I haven't tried this with more than three files. 

Maybe a counting semaphore of some type could be used to restrict the actual number of converts happening in parallel?  Or maybe there is one already... I'll start sifting through the code.