Mass Import plugin for cpg1.5.x

[S73]

is there anything I could do to avoid being logged out after a few minutes?

is there anything I could do for images to be assigned correctly to their albums?

is there anything I could do for albums to be properly assigned to their categories?

[S73]

Can anyone kindly simply point me in the right direction ?

Is it possible that the mass import tool is Not meant to be used in a bridged gallery but should be used in a native one?

I don't mind breaking the bridge and going back to native coppermine. But after the mass import, would I be able to re-bridge without losing the albums and categories?

[pols1337]

Can anyone kindly simply point me in the right direction ?

Is it possible that the mass import tool is Not meant to be used in a bridged gallery but should be used in a native one?

I don't mind breaking the bridge and going back to native coppermine. But after the mass import, would I be able to re-bridge without losing the albums and categories?

Hi S73,

I'm not an expert in Coppermine, but I got the 'mass import tool' to work for me in a bridged gallery (bridged with SMF2.0). 

If you do decide break the bridge and re-bridge, that shouldn't affect the albums or categories at all from what I understand.

And finally, I think you experienced the same problem as me when I first tried to run Mass Import -- the pictures got imported but were imported into a blank category.  As an admin, go to "Categories" and delete the blank category ... hopefully, that will un-hide the albums that were hidden in the blank category. 

And lastly, I would honestly recommend that you do Batch Add Files, simply because it's the safer method to add pictures.  I'm still testing out Mass Import Plug-in, and nobody supports the plug-in so you won't get any 'official' support on it. 

Cheers,
pols1337

[S73]

thank you loads pols and I highly appreciate your kind help.

Mine is bridged to vbulletin4 (the bridge says it supports vbulletin3).

I would have used the batch tool ...... but for 70,000 pics in 500 subfolders (albums) in 30 categories......... that is a loooooooooot of work to do manually.

I still thank you for your great feedback. I am breaking the bridge and trying now.

You know pols, I many times heard this " official support".

Well I never asked for official support. Any kind advice that helps me move a bit closer to my target is very welcome.

[pols1337]

I have 50,000 pictures in 400 folders.  Trust me, batch adding is not that bad  :-\

Also, breaking the bridge won't do anything.  I hope you understand that.  All that will do is modify the users and permission settings, but won't change how you add photos. 

[S73]

I have 50,000 pictures in 400 folders.  Trust me, batch adding is not that bad  :-\

Also, breaking the bridge won't do anything.  I hope you understand that.  All that will do is modify the users and permission settings, but won't change how you add photos.

Wow. you must be a very patient man. Or that you added those over some period of time.

I followed your advice, broke the bridge, and guess what...... things look much better now.

Categories were created properly, albums too, albums belong to their categories, and all the pics belong to their owner....... that is much better indeed.

I will let you know how it goes, but for the moment, 1000 pics were added to the right albums in the right categories........ hopefully. I will have to check thoroughly.

[DirtRider]

Ok I installed it but I just cannot get it to work at all. I create a folder under my albums folder called ftp. Now when I try and do the import it in the option to enter what folder it is or leave blank I put in ftp and I also tried leaving it blank. I then get this error

Critical error
There was an error while processing a database query

[Αndré]

Enable debug mode to get an extended error message.

[DirtRider]

Here we go

Code Select Expand


Critical error

There was an error while processing a database query.

While executing query 'INSERT INTO cpg15x_categories (pos, parent, name) VALUES ('10000', '0', '')' in plugins\mass_import\import.php on line 360 mySQL error: Field 'description' doesn't have a default value


File: C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\dasphoto\include\functions.inc.php - Line: 270





[Αndré]

Field 'description' doesn't have a default value

Either you adjust your database and give that field a default description or you modify the plugin code to submit the description.

[miblo69]

Hi,

There seems to be a problem with mass import when the directory names contain non-ASCII characters. In my case several directories contain swedish characters: åäöÅÄÖ. The import 'succeeds', but no images imported to the Albums that have Swedish characters in their names. Albums (directories) without characters are displayed correctly and contain the images as should.

I tried to locate the function in import.php, but I am no php coder.... If any one has tips on how to solve this, it would be greatly appreciated!

And, no, removing all Swedish characters in the directory names is not an option... I have some 80k pictures in 3000 directories I want to import.

Many Thanks!
~Mike

[Αndré]

removing all Swedish characters in the directory names is not an option... I have some 80k pictures in 3000 directories I want to import.

Would it be an option if the script renames your files and folders automatically before it adds them to the gallery?

[miblo69]

No, that wouldn't be an option. The whole Coppoermine Gallery suports UTF-8, so I would assume the Mass-Import would also support it.

I did some elementary troubleshooting, and I can see all the albums (including thoe with nonASCII characters) in the cpg_albums table. But the Pictures that reside in a non_ASCII directory, are not available in the cpg_pictures table. So part of the import seems to work. I.e Albums are ok, but not pictures. I would think there is a missing 'character conversion procedure' in the Picture import, but it is present in the Album import.

But my php knowledge limits me from troubleshooting this further...

Thanks,
~Mike

[Αndré]

Please try the following. Open import.php, find

Code Select Expand

$sql = "SELECT aid " . "FROM {$CONFIG['TABLE_ALBUMS']} " . "WHERE title='" . addslashes($title) . "' AND category=" . (INT)$category . " LIMIT 1";
and replace with

Code Select Expand

$sql = "SELECT aid " . "FROM {$CONFIG['TABLE_ALBUMS']} " . "WHERE title='" . addslashes(utf8_encode($title)) . "' AND category=" . (INT)$category . " LIMIT 1";

find

Code Select Expand

$sql = "SELECT pid " . "FROM {$CONFIG['TABLE_PICTURES']} " . "WHERE filepath='" . addslashes($dir_name) . "' AND filename='" . addslashes($sane_name) . "' " . "LIMIT 1";
and replace with

Code Select Expand

$sql = "SELECT pid " . "FROM {$CONFIG['TABLE_PICTURES']} " . "WHERE filepath='" . addslashes(utf8_encode($dir_name)) . "' AND filename='" . addslashes($sane_name) . "' " . "LIMIT 1";


This fixes the functionality in my testbed. However, it doesn't fix the output, but that's currently not important IMHO.

[miblo69]

HI,

No, it stil doesnt fix the problem. I removed the database completely, and did a fresh install. Copied my albums to import, to the /albums/ directory. Started Mass_import, applied your suggested changes, and it scrolled through all files. But unfortunately nothing was imported. It creates categories for each album, but they don't contain any pictures.

So the change has effectively stopped import of *all* albums. Previously it was only albums with non-ASCII characters in the directory name. Let me know what debug info you need and I'll provide it.

Thanks,
~Mike

[sindbad5]

I've installed the plugin. It creates categories and albums but will upload only up to 20 pictures per go. This is not helpfull  if there are some thousand images.

Tom

   

[Αndré]

Unattended run works for me as expected. Please wait some time when you see the window you attached as screenshot. I'm sure it will refresh and the number of remaining files will decrease.

[ge-flopt]

Hopefully you can help me. Have migrated mine coppermine site to a new disk. Have all the files, but forgot to transfer the database. I can't retrieve it, because the disk is already formated. Well hopefully mass import can help me. I have only one problem. The import tool gives me the following error:
Fatal error: Maximum execution time of 30 seconds exceeded in /......./plugins/mass_import/import.php on line 55

I can change the tomeout to a higher number, but i don't know if that's the correct way to go.

Any ideas?

Specs:
Server: Raspberry pi (it does the job!)
Images: 128.148 (Full files, Normal files and thumb files included)

THX!

[phill104]

Yes, that is the correct way to go. Try 60 seconds and gradually increase it until the script works.

[Αndré]

If I remember correctly the mass import plugin always imports a whole directory. So increasing the execution time might work for some directories, but maybe not for larger directories. I don't know if the plugin regards already existing intermediate-sized pictures and thumbnails. If so, the script just needs to add some meta data to the MySQL database. If you're hosting the database (and files) on the SD card, it's maybe a good idea to transfer them to a HDD to increase performance.

You maybe also want to run the import process on your desktop PC, which has usually much more power, and then export/import the database dump when completed. So you avoid any PHP timeout issues.


OT: I always wanted to install Coppermine on one of my Pis (early model B with 256 and 512 MB RAM) and check how it performs, but never done that. Do you use a Raspberry Pi 1 or 2? How's the performance / Coppermine page load times?