Mass Import plugin for cpg1.5.x

[Joachim Müller]

Mass Import gives the admin the ability to import large numbers of pictures organized by directory structure. The plugin is a port of the Mass Import plugin for cpg1.4.x.
The mass import works similarly to the batch-add process, but it allows you to add an entire structure of folders, subfolders and files to be added in one go. The plugin will create categories and albums that correspond to the folder names. It will then loop though the files in the structure and batch-add them to the database and create the resized images.
Use this plugin as well if you have issues with the regular batch-add process consuming too many resources.
You can download the tarball from the SVN: http://coppermine.svn.sourceforge.net/viewvc/coppermine/branches/cpg1.5.x/plugins/mass_import.tar.gz?view=tar if you know your way around. To download the plugin as a zip file package, click here: http://sourceforge.net/projects/coppermine/files/Plugins/1.5.x/cpg1.5.x_plugin_mass-import_v3.4.zip/download
Attached is a screenshot of the plugin "in action". Due to the nature of the plugin (it is admin-only) there is no demo available. The plugin will add another admin menu item named "mass import" next to the batch-add button.
Please understand that I will not support you in the usage of this plugin, so it's sort-of "take it or leave it".

Here's the content of the readme file that comes with the plugin:

Code Select Expand

/**************************************************

  Coppermine 1.5.x Plugin - mass_import

  *************************************************

  Copyright (c) 2010 Nibbler

  *************************************************

  This program is free software; you can redistribute it and/or modify

  it under the terms of the GNU General Public License as published by

  the Free Software Foundation; either version 3 of the License, or

  (at your option) any later version.

  ********************************************

  $HeadURL: https://coppermine.svn.sourceforge.net/svnroot/coppermine/branches/cpg1.5.x/plugins/mass_import/readme.txt $

  $Revision: 7149 $

  $LastChangedBy: gaugau $

  $Date: 2010-01-28 10:17:02 +0100 (Do, 28. Jan 2010) $

  **************************************************/

What it does
============
Mass Import gives the admin the ability to import large numbers of pictures organized by directory structure.

Details
=======
The mass import works similarly to the batch-add process, but it allows you to add an entire structure
of folders, subfolders and files to be added in one go.
The plugin will create categories and albums that correspond to the folder names. It will then loop though
the files in the structure and batch-add them to the database and create the resized images.
Use this plugin as well if you have issues with the regular batch-add process consuming too many resources.

Announcement
============
Visit the announcement thread of this plugin for breaking news
http://forum.coppermine-gallery.net/index.php/topic,61281.0.html

SVN checkouts
=============
Use subversion checkout to get the latest and greatest version of this plugin. Refer to the documentation
that comes with Coppermine to find out details on the Subversion repository.
Check out https://coppermine.svn.sourceforge.net/svnroot/coppermine/branches/cpg1.5.x/plugins/mass_import/
or use WebSVN to browse the repository at
http://coppermine.svn.sourceforge.net/viewvc/coppermine/branches/cpg1.5.x/plugins/mass_import/

Credits
=======
Nibbler initially created this as a mod of the batch-add function that featured the delayed execution.
Donnoman converted the mod into a plugin for cpg1.4.x
Flux and Paul Van Rompay contributed code to improve the functionality.
Joachim Müller ported the plugin for cpg1.5.x and spiced up the output a bit.

Support
=======
The scope of support for this tool is limited. Basically, it comes as-is.

Todo
====

The changelog's content:

Code Select Expand

/**************************************************

  Coppermine 1.5.x Plugin - mass_import

  *************************************************

  Copyright (c) 2010 Nibbler

  *************************************************

  This program is free software; you can redistribute it and/or modify

  it under the terms of the GNU General Public License as published by

  the Free Software Foundation; either version 3 of the License, or

  (at your option) any later version.

  ********************************************

  $HeadURL: https://coppermine.svn.sourceforge.net/svnroot/coppermine/branches/cpg1.5.x/plugins/mass_import/changelog.txt $

  $Revision: 7149 $

  $LastChangedBy: gaugau $

  $Date: 2010-01-28 10:17:02 +0100 (Do, 28. Jan 2010) $

  **************************************************/



Changelog

=========

[A] = Added new feature

[B] = Bugfix (fix something that wasn't working as expected)

[C] = Cosmetical fix (layout, typo etc.)

[D] = Documentation improvements

[M] = Maintenance works

[O] = Optimization of code

[S] = Security fix (issues that are related to security)

*********************************************


2010-01-28 [M] Counted plugin version up from 3.3 to 3.4 {GauGau}
2010-01-28 [A] Added Finnish language file {GauGau}

2010-01-28 [M] Added changelog {GauGau}

2009-08-20 [M] First public release of the plugin for cpg1.5.x (plugin version 3.3) {GauGau}
2009-08-20 [D] Added screenshot to install information {GauGau}
2009-08-20 [O] I18n of configuration file {GauGau}
2009-08-19 [A] Addeded spin button {GauGau}
2009-08-19 [A] Addeded German language file {GauGau}
2009-08-19 [A] Added menu icons {GauGau}
2009-08-19 [C] Converted admin menu entry function for cpg1.5.x {GauGau}
2009-08-18 [B] Inspectified form {GauGau}
2009-08-18 [C] Spiced up form {GauGau}
2009-08-18 [M] Added SVN header {GauGau}
2009-08-18 [M] Renamed plugin folder from CPGMassImport to mass_import to respect naming conventions {GauGau}
2009-08-18 [M] Increased plugin version count from 2.0 to 3.0 to reflect the in-depth changes {GauGau}

[papukaija]

Great plugin. The Finnish translation is added to this message.

[sammyd109]

When using this plugin to import file structures that contain several level of folders, what sort of structure of albums or categories will be used? Will they follow the folder structure or will they simply import the pictures into a mass album?

http://ec2-75-101-214-77.compute-1.amazonaws.com/login.php

[sammyd109]

The login info for this site is as follows:

email: test@test.com
password: **** [/Edit Joachim] Edited out the password of the admin account you posted [/Edit]

[Joachim Müller]

That's an admin account. Posting an admin account publicly is silly. I have logged in and changed the password of that account. I have edited your posting above. Remove that account from your gallery.
The mass import plugin is not installed in your gallery, so what should the test user account do? Don't expect to be spoon-fed. Stop wasting our time - all you're doing is causing extra moderation effort.

[panic83]

Hi,
Coppermine Photo Gallery 1.5.8 (stable)

Gallery Configuration - File settings - Auto resize images that are larger than max width or height

When i set this settings to "NO", i got error  in "Mass Import".

settings:
http://img196.imageshack.us/img196/6343/screenshot1vx.png
error:
http://img214.imageshack.us/img214/5506/screenshotitp.png

Please advice.

I just want to add photos form folder without  any convention.

[panic83]

without  any convention.

without convertion of course, sorry for typo

[justforthispost]

First of all I want to say thanks for this wonderful plugin. I've used it to add more then 250 albums to my gallery!

There is however 1 little thing that is bothering me. When I manually create an album the user permissions on the album are set to: uploads no, comments yes and votes yes. On the support forums here I found a way to chance the upload permissions to yes due to a modification in the delete.php file. This works like a charm when manually creating albums.

But if I add the albums with the Mass Import plugin the permissions aren't set as I want to. Is there a way that you give us the option to chance the permissions in a new release of Mass Import?

Kind regards.

[Αndré]

Is there a way that you give us the option to chance the permissions in a new release of Mass Import?

Please understand that I will not support you in the usage of this plugin, so it's sort-of "take it or leave it".

So you have 2 options:
1. Change all settings with an SQL query
2. Modify the plugin code

[justforthispost]

Thanks Αndré.

I'll go for option 1 then.

[knym]

First of all I want to say thanks for this wonderful plugin. I've used it to add more then 250 albums to my gallery!

There is however 1 little thing that is bothering me. When I manually create an album the user permissions on the album are set to: uploads no, comments yes and votes yes. On the support forums here I found a way to chance the upload permissions to yes due to a modification in the delete.php file. This works like a charm when manually creating albums.

But if I add the albums with the Mass Import plugin the permissions aren't set as I want to. Is there a way that you give us the option to chance the permissions in a new release of Mass Import?

Kind regards.

While your there, and anyone who reads after and finds the same delema.. load your servers mysql database editor or install something like phpmyadmin.. locate in the database table albums .. default install being "cpg15x_albums
" and inside there find the field uploads.. it's an anum field that accepts YES or NO.. The default value is NO.. simple change it to YES and from that time forward YES will be selected as default instead of NO.  Note you can change the defaults for the other two fields (comments and votes) mentioned in post as well. Those are default set to YES.

All future problems solved..

[ls5302]

I have just upgraded to Coppermine version 1.5.12, and installed the Mass Import plugin. It looks like with MySQL 5.5.10 the description field in the categories table cannot have a default value of '', yet a couple of inserts statements in import.php fail to assign an explicit value. I'd like to suggest, if I may, the modification of the following lines in import.php:

Line 168:
$db_query_wrapper("INSERT INTO {$CONFIG['TABLE_CATEGORIES']} (pos, parent, name, description) VALUES ('10000', '$parent', '" . addslashes($name) . "', '')");

Line 360:
$db_query_wrapper("INSERT INTO {$CONFIG['TABLE_CATEGORIES']} (pos, parent, name, description) VALUES ('10000', '0', '{$post_directory}', '')");

Great plugin!!!

[ls5302]

Cannot edit post and missed one change:

Line 189:
$db_query_wrapper("INSERT INTO {$CONFIG['TABLE_ALBUMS']} (category, title, pos, description) VALUES ('".(INT)$category."', '" . addslashes($title) . "', '10000', '')");

[Αndré]

I haven't checked the behavior of MySQL 5.5.10, but committed your changes in svn revision 8209. People who have the same issue as ls5302 should use the latest svn version of the plugin until no new version is released.

[S73]

The mass import works similarly to the batch-add process, but it allows you to add an entire structure of folders, subfolders and files to be added in one go. The plugin will create categories and albums that correspond to the folder names. It will then loop though the files in the structure and batch-add them to the database and create the resized images.

Oh Good Lord!

Have you been reading into my mind ?

[S73]

Hallo Joachim, Bonsoir André,

the tool worked very nicely on my gallery and properly imported all subfolders as categories...... However it did not import any one single image......

any idea?

[S73]

  I even contacted my support and asked them to check the error logs.

Curiously, there's nothing in the server's error_logs either.

You would expect to see something if there were a permissions error.

Is it possible that because I have about 500 subfolders with about 70,000 files in all that the script hit a resource limit (such as a memory limit) while enumerating the files, and simply died before it could complete ?

[S73]

I figured out that memory was not enough. I regret your script did not display clear crash messages (maybe actually that was in the collapsed debug in the footer.... )

Anyway, memory was increased and script is working now.......

One very strange thing though.....

every single picture added belongs to no album.

If you look in the database, all the imported pictures belong to album 0 (aid=0)

Any idea why ?

[S73]

and all albums belong to category 0 ..... 

[S73]

and while the images are being added in batches of 10, I am regularly disconnected with this message:

Error
You don't have permission to access this page.

But I was logged in as admin and the mass import tool was working........ why am I then still disconnected ? There is no inactivity there at all as the mass import refires itself every 1000 millisecond.