[Closed]: Trouble mass adding files

[ToTheMax]

I will have approximately 3,000 files to upload and import daily. I have access to my Server so I can move the file instead of FTP'ng them.
If all goes well, the site will have millions of files. Is that a problem?
Because of the large amount of files I get daily, I need a way to import them unattended. I'm in a controlled environment so there are few concerns about the files. Every one of them will be .jpg

I can use Batch add files but it times out and I have to reload the page several times.
Q: Can I change the settings so the page won't timeout?
I uploaded the Mass Import Plugin. It looked like the perfect solution. It creates a directory structure but won't import the files. I get "There was an error while processing a database query". I tried Timer.php and got the same error.
Cpg-contrib.org. is down and has been for quite a while.

Can anyone help or direct me to a place to get help?

I would send the link to my site but I'm behind a firewall.

[ToTheMax]

Sorry,
I'm using CPG version 1.4.26, PHP 5.2.13, mysql 5.051a, Windows XP IIS for my test enviornment.
Will send results of debug if needed.

[phill104]

Your server timing out is a setup problem with your server which only you can fix. You need to find out why and that is something we cannot advise on.

Yes, we know cpgcontrib is down and as said befoer there is nothing we can do to get a third party site back up. Again, with that plugin we cannot help as it appears to be a setup problem on your system. I am guessing you are self hosting? That means you have only you to turn to for support, sorry.

[ToTheMax]

The reason I asked in this forum is because several of the authors of the Mass Import Plugin were admin's or contributors of CPG.
author='Nibbler, Donnoman <a href="http://cpg-contrib.org" target="_blank">cpg-contrib.org</a>, Flux, and Paver.';
I also wanted to know if you knew if there would be a problem having millions of files in CPG? I will have terabites or storage.
As far as the timeout issue, I increased the settings in PHP.ini to 360 and increased the timeout on IIS. There was a bit of improvement.

Thanks for your help. Guess I'll have to try something else.

[Joachim Müller]

There is no restriction built into coppermine, but having millions of files definitely won't work without in-deep changes in coppermine's core structure. If you need an application that can handle millions of files and only want to use it internally on an intranet, you better look for another solutions.
Has been asked and answered before btw.
Just increasing the single timeout parameter in php.ini is wishfull thinking. Sounds like you don't have the skills to be self-hosting such a huge gallery... Just buying a hard drive with several terrabytes is not what qualifies you to perform self-hosting... . I'm not trying to be rude, those are just facts, sorry.

[ToTheMax]

Oh nevermind. I wont grace your insults with a reply.
I will say I found exactly what I was looking for, somewhere else.

[phill104]

Please share your wisdom.

[ToTheMax]

Please share my wisdom??? You must be kidding!
Maybe you restrained yourself and only half insulted me.

I said the page was timing out and immediately the assumption was it must be a setup issue. Nevermind that everything else worked correctly.
Even the Batch Add Files included with CPG has a line that says "If your browser timeouts, hit the reload button". What a resourceful solution.

I loaded the Mass Import plugin on the recommendation of Donnoman and when it didnt work I was looking through the code and noticed the authors names. There was a nibbler, donnoman, flux, and paver. I had seen these names on this forum and since cpgcontrib was down which I mentioned in case you were unaware, I thought what better place to seek expert advise other than with the authors.

I never mentioned I was on an Intranet yet somehow someone jumped to that conclusion. Actually I intentionally left that part out because I didnt want anyone to think that was the case. All I said was that I was behind a firewall and I was running Windows XP IIS for my "test enviornment". A router is a firewall of sorts.

I said my TEST ENVIRONMENT yet it was immediately assumed that I was self hosting. But I suppose on one has ever used a test environment to determine if a certain software meets their need before putting it in place. There must be a few people that would put in some front end work to save time on the backend. And having to sit through importing each file is truly torture.

I spent several days (yes, I really did) searching this SUPPORT site looking for something to help before I posted. But there is a limit to the keywords you can think of and a limit to reading where others got insulted because they failed to read or understand the section that pertained to their problem. Its difficult to know where to look for an answer when you barely know or understand the problem.

How can anyone determine from such a small amount of info what skills I have or what kind of support team I have or am a part of? Amazing how someone can determine that just because I have a terabyte HD that I bought it for this application or that I'm self hosting.

I would think many of your serious users would reach the million files mark sooner or later. I saw where a few users asked a few questions about the capacity of cpg and were told drive space was the determining factor.

What I gather from this is Dont ask the Coppermine experts in this Support Forum. If you do, expect the solution from another user first.
But do expect to be insulted by the experts and have more keystrokes devoted to telling you to RTFM than to answer the question.

My last plea on my first post was "Can anyone help or direct me to a place to get help?" I was told what wont work, what I wasnt capable of, how my setup wasnt correct, etc.
Everything except help or directions to find help.

Feel free to lock, delete, move or whatever to this thread. And save the keystrokes in responding to me, I shall not be back here again.

[Joachim Müller]

And save the keystrokes in responding to me, I shall not be back here again.

OK, fine. Farewell.

In case you do return and read this: you claimed to have found the answer to your question (somewhere else) and Phill asked you to share your insight with others for the benefit of others with similar issues to yours. Yet you decided not to tell us what solution you found and instead posted a useless rant.
I leave it to the decision of the visitors of this thread wether that's what they consider correct behaviour on a support board.

All I said was that I was behind a firewall and I was running Windows XP IIS for my "test enviornment". A router is a firewall of sorts.

A-ha. You're an expert. Sorry, I was wrong in my initial posting: you must be a genius. Real expert talk .

Even the Batch Add Files included with CPG has a line that says "If your browser timeouts, hit the reload button". What a resourceful solution.

Yes, I think that this is a good way to code that file - it's pretty nifty design to make the code not to attempt to process the files from scratch, but only those that have not been added already to the database. Take a look at other apps, where this is not the case. This is a feature. You're just not clever enough to realize that.

I would think many of your serious users would reach the million files mark sooner or later.

Sure. Ask mySQL AB how their database handles millions of records. I guess they will advice you to use oracle instead.

Feel free to lock, delete, move or whatever to this thread.

Thanks for granting us permission to do so. You're really a generous person. Marking thread accordingly (as you failed to do that).