Batch add with automatic album/cat structure from directory tree.

[Nibbler]

Here we go, upload this into your coppermine directory and call it timer.php (I hardcoded it, so sue me ) then go there, you know the drill.

It will keep reloading until all the pics are added if you set it to, so you can start it and leave it unattended for hours until it finishes, even with php set with a very low max_execution_time (ignore the fatal error). Not recommended for shared hosting environments unless you are vewy vewy careful.

Note: Auto reload needs js support.

[Note: Tagged as 1.4, see later posts in thread for updated versions of this mod]

[mrplow]

You, sir, are a legend.



I thank you for your sterling work.




I can't express how awesomely happy I am with this hack! 


edit (heh..)
EXCEPT... it's a shame it can't resume a folder/cat... if I bung a load more files into the same folder then ask your amazing script to do its work again, it starts a new catagory and populates it with new albums, all of the same names as before, and starts from scratch.
Shame about that

[thumeau]

Thank you so much Nibbler,

So glad to be able to batch input more than 1 album at a time.

I can't get to have the autorun / auto reload working eventhough JS is definitely on.
I tried both on Mozilla Firefox and IE...
Any ideas?

Thanks,
Thomas

[oshelef]

I get the following error when I try to run this:


"Created root category

Warning: Invalid argument supplied for foreach() in /var/www/html/coppermine/timer.php on line 200
structure created, 0 files to add."

The category is created, but no albums or pictures are included. (I've gotten more complicated errors before, but I got the above trying to use it add a single image, in a single directory)

I.e. directiory albums/test contains test.jpg and nothing else. permissions are all 775.

Any suggestions as to what might cause this?

Thanks,

Orie

[Nibbler]

OK,  you make a folder in albums -> that becomes the root category.
You cannot then have any pics directly in that folder since you cannot have pics directly in a category.
You need to make folders inside the root folder - these become your albums, or more categories.

Bottom line is - You can only create a directory/file structure that would be a legal category/album/pic structure in coppermine.

[oshelef]

Ahh, I think I understand it. I need a folder structure that looks like:

Category/Album/Image.jpg right?

I redid my test and it worked; although it still spit out an error, it did insert my test image as it should.

Edit, it works with one image, but when I have more than one image I get this following error:

"Fatal error: Call to undefined function: array_diff_assoc() in /var/www/html/coppermine/timer.php on line 98
structure created, 700 files to add."

Thanks for your help.

Orie

[Nibbler]

Ah, seems the script requires PHP 4.3.0 or better to work. You can try adding the substitute function in the first comment here into the script, or you could upgrade your PHP, or you could try using PHP_Compat.

[donnoman]

Nibbler - Great script.

"It needed more power; so I rewired it"  (Tool Time Joke, hope somebody gets it)

I modified the script so that #1 you can run it against the album directory itself as the root cat.. There's an exclusion list that eliminates the basic garbage folders you might find there.

#2 It eliminates trying to add thumb_ and normal_ images that are already there from previous batch attempts.

#3 Checks to see if it can use a pre-existing cat/album before creating the new ones.

Which allows you to run this to do your batch adds all the time, successive runs don't hurt anything.

I was using this script against 1.4, but I didn't change anything that would be materially different than 1.3 so I would expect it to work on both platforms.

Also a nice little side effect of running this with 1.4 is that IPTC titles,keywords, and comments are automatically imported if you have iptc turned on in the config.

[donnoman]

@mrplow With my changes there should be no problem resuming. Do it as many times as you like.

[Juniors]

Edit (double Post)
......

[Juniors]

Hello

With both Scripts i get the following Error when i call the timer.php from my Browser:

Parse error: parse error, unexpected T_ENCAPSED_AND_WHITESPACE, expecting T_STRING or T_VARIABLE or T_NUM_STRING in /abc/def//timer.php on line 174

Any hints?

Thank you...

[donnoman]

im sure the line number that errors out isn't the same between the two scripts. make a note of which script gives which line number.

[kegobeer]

This is probably a problem with the file format.  I've attached a plain text version of Donnoman's file.  Try this one and see if you still get those errors.

[Juniors]

Hi Kegobeer

Thank you for the Script. With this one i recieve the following Error:

Parse error: parse error, unexpected T_ENCAPSED_AND_WHITESPACE, expecting T_STRING or T_VARIABLE or T_NUM_STRING in /abc/def/timer.php on line 416

Any hints again?

Junior

[Nibbler]

That file is 266 lines long. Therefore there cannot be an error on line 416. I suggest you try a different ftp client.

[Juniors]

Hi

Iam sorry. I fixed the Script. Now its exactly 266 lines long and my Error-Msg is:

Parse error: parse error, unexpected T_ENCAPSED_AND_WHITESPACE, expecting T_STRING or T_VARIABLE or T_NUM_STRING in /abc/def/timer.php on line 192

Line 192:

Code Select Expand

$auto = (isset($_POST['auto']) && $_POST['auto']) ? 'checked = "checked"' : '';

Code Select Expand

[kegobeer]

Are you uploading your files as ASCII or binary?  These files need to be uploaded as ASCII, or use the auto switch so your FTP client will do it for you.

[Juniors]

I uploaded the filles in ASCII of corse 

[kegobeer]

Then perhaps you should try a different FTP client.

[flux]

Hi,

here's my small participation to this nice and very usefull script..

Unfortunately as i didn't read the whole topic and especially donnoman's posts i didn't see that some of my modificatons had already been introduced and i just grabbed the file at the top of the topic, expecting it to be the latest version.. i was quite in a rush and started to make my modifications right away, silly me

Here's how the script works :
The script expects a directory name under ./albums/, that directory is supposed to be a root category in your gallery. Bottom directories, that should contain your pictures, will be treated as albums and intermediate directories will be treated as sub categories from the root category of course..
here's a quick example, let's say you have the following root category tree :
./albums/lib/
you should declare "lib" as the root directory to process.
now if you have the following tree under lib :
./albums/lib/A/Aname/pic.jpg

a root category named "lib" will be created, a sub category "lib>A", name "A" will be created, an album "lib>A>Aname" named "Aname" will be created and the picture pic.jpg will be added to the album "lib>A>Aname"..

that's how the original script works..

Now here are the features i've added to the original script :

In the same way donnoman did it, i added some filtering in order to avoid adding the same category/album/picture twice. So if you run the script twice on the aboce dirctory tree, everything should be created/added the first time and nothing will be done the second time..

Beware that there is a "limitation" in my database query on how category and album names are matched. Indeed a side efect of the query will be to limit creation of categories and albums to unique names. Which is not a limitation in the orignal design of CPG but is obviously a limitation of the filesystem (you cannot have the same directory name twice at the same location).. So you cannot have two categories or albums with the same name at the same location.. For example if you manually create, thru cpg interface, a second "Aname" album under "lib>A" and run the script to add new pics located in "/lib/A/Aname/" the script will use the first ID of the albums named "Aname".. maybe it's not worth mentioning, i don't know.. The same thing will happen with Category Names..
Of course if your second "Aname" album is located under "lib>Amisc>" then it's not a problem as the script will always use the reference to the parent when it tries to find if an album or category name already exists.. usually different parents mean different children


Then the pictures filter was already there in the original script but was applied while processinf every pictures found.. I've add to that filter a global pictures filtering that will remove from the pictures lists build on the directory tree, the pictures already inserted in the database.. So for example if you update the directory tree under"./albums/lib", let's say it contains 20k+ pics but only have 150 new pictures, then my script will purge the already added 20k+ pics and add directly the 150 new pics.. quite nice isn'it 

A thing you shoul also know it that i've modified the way filenames are build.. In the original script any char not being part of the lists a-z A-Z 0-9 would be replaced by the char "_" (underscore).. I've improved that by removing existing double underscores "__" until there aren't any doubles left..

You'll probably notice that the renaming process will crush any non US chars, so you might want to modify that if you filenames have international chars in it..

Well i think it's all i can say about it..
I've tested it against a quite huge fake directory tree made of 26 subcategories (a-z) and 26x26 albums (26 albums in each 26 subcategories), each album containing 50 times the same picture named from 1.jpg to 50.jpg, which make a total of 33800 pictures..

The script take a few seconds (25sec) to build and filter the directory tree, which is located on a remote nfs server, then it will process the pictures according to the reloading parameters you've chosen.. The first update took a few hours as expected.. Then adding a few pics here and there just take a few seconds, no need to go over the previously added 30k pics right ?

Thx for the coder of the original script..
Enjoy,

flux