Having a problem with uploading images. My web hosting co. shut me down.

[photoguy1]

I have read the rules for posting, but I don't know how else to get around saying..Please help!" (sorry Joachim if I'm not posting properly). My main problem right now is that my web hosting company (Lunarpages) has shut my website down and it's been over a week. They say I'm using too much resources on a shared server. I believe it's when I was attempting to upload 100's of 6mb images at once. They shut me down once before and it was determined that if I put a delay in the uploading, that would solve the problem. It did, but I think there also needs to be a delay in the amount of images uploaded at one time.

This is what I'm asking someone to help me with:
1. To look at my website (I would give you my login and password) and see the scripts that were written and how my site is operating, and how improvements can be made so that I'm not shut down again.
2. Help me determine if I can, in fact survive on a shared server or if I need a dedicated server. I believe I can work on a shared server if my site is configured correctly.
3. I need someone to help in the design of my website. I have specific ideas as to what I'd like and I need someone to tell me if it's possible.

I'm enclosing the comments from LunarPages about the resources I'm using ad where I exceeded them. But, I don't think these numbers will mean anything until you go into my site to see how I'm spiking and how it can be fixed:

(E-mail reply from Lunarpages)
"Acceptable usage range for our basic shared plan is 1.0% average CPU% or less, 1.0% average Memory% or less, 0.2% average MySQL% or less, and 20% Top Processes or less. We use the resource usage statistics as generated by WebHostManager. These statistics are a 24 hour cumulative average so there spike likely happened over a period of 24 hours. The Top Processes lines only indicate the script that is found to be most active on the CPU. Since index.php is the script called initially by all visitors it is not surprising that it is listed. However, this does not indicate that index.php is the source of the issue. index.php can be running very frequently and consuming minimal resources while another script may run less frequency but consume more resources. At the time of the script being disabled the usage statistics were reported as: Average CPU usage - 3.81% Average Memory usage - 1.07% Average MySQL usage - 0.2% rings3 ringsidephotos.com 3.81 1.07 0.2 Top Process %CPU 36.3 /usr/bin/php /home/rings3/public_html/coppermine/index.php Top Process %CPU 34.9 /usr/bin/php /home/rings3/public_html/coppermine/index.php Top Process %CPU 29.9 /usr/bin/php /home/rings3/public_html/coppermine/index.php You were using 3.81% cpu (we allow 1%), 1.07% memory (we allow 1%), 0.2% mysql (we allow 0.2% - you were good here), and top processes of 36.3%, 34.9%, and 29.9% respectively (we allow 20%). Checking your logs is a good way to try to find out what is causing the issue. In your webalizer logs, a displayimage.php script is called often: Top 30 of 16604 Total URLs # Hits KBytes URL 1 48597 4.43% 1608857 9.15% /displayimage.php 2 25649 2.34% 853148 4.85% /coppermine/displayimage.php 3 16528 1.51% 1007076 5.73% /thumbnails.php 4 8876 0.81% 1391072 7.91% / 5 7914 0.72% 308 0.00% /images/spacer.gif Top 10 of 16604 Total URLs By KBytes # Hits KBytes URL 1 48597 4.43% 1608857 9.15% /displayimage.php 2 8876 0.81% 1391072 7.91% / 3 5074 0.46% 1051729 5.98% /plugins/ringside_photos/js/jquery.ui.all.js 4 16528 1.51% 1007076 5.73% /thumbnails.php 5 25649 2.34% 853148 4.85% /coppermine/displayimage.php Here are the most common causes for high usage: - your scripts are not upgraded: you may have not upgraded your scripts yet, as older versions of the scripts can have bugs in them that would cause high CPU and Memory usage - your scripts are configured incorrectly: you might need to check your scripts` configuration for any tweaks of settings available that might influence the usage, as a single wrong setting might do a whole lot of harm - your plugins are misbehaving: some plugins might cause high load because they were badly written. You might consider disabling them or finding alternatives for some plugins - bots are flooding your site: your pages might have been found by spam bots that try continuously to send messages via your contact form, or add various comments to your items and such, depending on your site's configuration. An implementation of a CAPTCHA system would be useful to keep bots out of your pages, while checking for the highly accessed webpages using the tools provided by CPanel. - you might need caching: a caching plugin will help in most cases as it will serve pre-generated html pages instead of using the resource intensive PHP scripts to generate the same page over and over again for different visitors of the site."

I have now had three people attempt to help me with my site over the past few years. The first two totally messed up my site. The third is a nice guy and has helped a lot of the past few years, but it's now over three years later and he still hasn't finished designing just my home page, let alone all the other things I wanted done. One error made was when he was configuring something on my site, he forgot to protect my images (full size images to download which needs a password and login) and hundreds of images were "taken", which cost me thousands. My site has been shut down for over a week and I'm just about out of business. I cannot afford to have someone "tinker" with my site. I am asking for someone fully knowledgeable of Coppermine and making the changes I need and want. I also cannot have someone make changes every few months or when they get around to it. I need work done immediately.

The only way to fully understand what to do, is to go inside my site, look at all the stats and errors that caused the shutdown. I think Coppermine is a phenominal program and my hat off to all those that helped create it, but for someone who doesn't have the time to learn it, I am at the mercy of someone until they get around to fixing the problem.

Again, please no newbies reply that want to "try" and figure out the problems. I need a pro. I hope that's possible because if my site is down any longer, I won't need any help at all. I'll just need to find a new job!

Marty   

[photoguy1]

My website is: www.ringsidephotos.com

[Gene-2008]

I'm not prepared to help you to the extent your asking for but maybe a few questions would help others to decide.

Are you trying to sell your images and that I why you are posting such large images?  I don't see a shopping cart...anywhere?

If you are just showing images (and/or taking orders which you will fulfill separately) then I believe you could reduce significantly your server demand by sizing you photos for web viewing.  What you are uploading as a 3-6MB file with display resolution of 2000x3000 pixels could probably be view as 533x800 and be only about 300KB.  I would guess that some of your CPU usage is also related to have to create thumb nails and intermediate images for your large files.

So I guess what I'm wondering is the why of such large images...hope that helps some...If it doesn't it didn't cost you anything.

Gene
BTW I could help you with working out layout and theme issues...yours looks pretty strange to me.

[photoguy1]

I am a professional boxing photographer and I do sell these images. That's why they need to be that large. Clients log in and have access to download the full size image. All other guests can view the thumbnail and intermediate images, but cannot download the full size image. Also, there's no shopping cart because I bill magazines and websites by looking at their download history.

As far as the design, it's taking the last person three years (I'm an idiot for waiting that long) just to work on the home page and it's only half done. The left side of the page is okay, but the right side is plain and not what I want. I want the home page filled with photos from different galleries (Boxing Galleries, Press Conferences, etc.) and have slide shows going. I purchased a web design package and wanted it incorporated in with Coppermine. I'd like to be able to easily insert new photos within each box or slide show. I need things simple, otherwise, I'll never get around to changing images.

As I mentioned, I've had three people work on my site with no clear results. They just write script over script and my site gets messed up after a while. 

Marty

[Gene-2008]

Seems to me you need to revamp the website/process...let Coppermine manage the display (and selection) of photos that are "websized" and then have a purchase script that packages the fullsized images into a downloadable zip file for downloading.  Thus never showing/manipulating the large images.  As for as the Website those things are much more straight forward.

You can PM me if you'd like to discuss more.

Gene

[Joachim Müller]

I understand perfectly well that you can't reduce the quality of the pics (resolution-wise) because you need your customers to be able to download hires images, so the answer to your question is pretty straightforward imo: as your webhost is acting up on your resources consumption you have exactly two options:

• Resize the pics on your client before uploading them, i.e. create the intermediate image and the thumbnail on your client (respecting the aspect ration conversion and the naming scheme) and FTP upload all files (orginal, intermediate and thumbnail) to your server, then perform the batch-add. As a result, coppermine will not attempt to perform the resources-intensive resizing of images and will just populate the database with your newly added pics (if the images already exist). This batch-resizing can be accomplished on the client-side easily using one of the many tools that are freely available. Take a look at http://coppermine-gallery.net/tutorial/irfanview1.php for example
• Improve your webhosting contract
  You appear to make a living out of your site, selling your pics, so why do you have such a lousy shared webhosting contract if that is causing all your issues. Either discuss with your existimg webhost what improved server options they can offer or find another webhost where you will be hosted with less domains on one physical server. In an ideal world, get a dedicated server. I just googled for that quickly - you should get a true dedicated for around $50/month and a virtual dedicated machine for $30/month in the US

There are imo no alternatives to those two options. Decide for yourself wether the extra work of resizing on the client is worth 30 bucks a month.

Joachim

[photoguy1]

Joachim,
Thank you for responding. To be honest, I've looked at the tutorial twice and I don't understand how the client gets the full size image. This shows how to batch re-size and right there I have a slight problem. I size my images in two different sizes (8x10 and 8x12). I have a batch resizing in Photoshop, but never use it because I crop in the two different sizes. Again, please forgive my ignorance in these matters. Plus, the fact that my site has been down for over a week, I really can't think straight. Add in the fact that the last person who was working on my site took three years and still hadn't finished my home page, let alone all the other pages!!! I also realize that it is his mistakes that keep getting me shut down. Someone (Gene-2008) responded to my post. Would you recommend I have him apply this program? Just to be sure, this will allow me to have thumbnail and intermediate images displayed, without using up all the resources on the server, but when a client needs to download a full size image (with login and password only), they can do it 24/7? That was the reason I have all these full size images on the server.

As far as dedicated server, the fact that I am using 140gb's of space, the plans I have found are more than $100.00/month. Can you please forward the few sites you saw for a dedicated server? If you say you have the solution, please tell me how I can proceed. I am not knowledgeable enough to do these things myself...Sorry!

Marty

[Gene-2008]

Note all comments/questions here are meant to help...take them for what they are...

Joachim,
Thank you for responding.

To be honest, I've looked at the tutorial twice and I don't understand how the client gets the full size image.

What Joachim is suggesting here it that part of the server utilization comes from the server having to create
the thumbnails and intermediate images.  If you batch resize on your local PC that will eliminate that utilization completely.
You would then upload three files for each photo: thumbnail, intermediate, and a full-size.
There are several programs that can do the resizing...

This shows how to batch re-size and right there I have a slight problem.
I size my images in two different sizes (8x10 and 8x12).

Why are you providing 8x10 and 8x12 surely your clients can crop down to 8x10 if you give 8x12?

I have a batch resizing in Photoshop, but never use it because I crop in the two different sizes.
Again, please forgive my ignorance in these matters.
Plus, the fact that my site has been down for over a week, I really can't think straight.

Add in the fact that the last person who was working on my site took three years and still hadn't finished my home page, let alone all the other pages!!!
I also realize that it is his mistakes that keep getting me shut down.

Someone (Gene-2008) responded to my post. Would you recommend I have him apply this program?

I don't think you'll get much comment on this...Joachim doesn't know me and probably
would very hesitant to recommend anyone. But he can comment if he'd like... :-), 

As for having me do it.  It is not difficult:
You determine your required thumbnail size and batch create files in that size and name them thumb_FILENAME.jpg
You determine your required intermediate size and batch create files in that size and name them normal_FILENAME.jpg
And then you upload the three files thumb_FILENAME.jpg, normal_FILENAME.jpg, and FILENAME.jpg

Just to be sure, this will allow me to have thumbnail and intermediate images displayed,
without using up all the resources on the server, but when a client needs to download a
full size image (with login and password only),
they can do it 24/7? That was the reason I have all these full size images on the server.

As far as dedicated server, the fact that I am using 140gb's of space, the plans I have found are more than $100.00/month.

So you are saying you have ~14000 images online and people look back that at them and
purchase them?  Just wondering...whether some of them could be moved offline...again just wondering?

Can you please forward the few sites you saw for a dedicated server?
If you say you have the solution, please tell me how I can proceed.
I am not knowledgeable enough to do these things myself...Sorry!

Marty

Another point that Joachim is making is if this is required to do your job why are you worrying about $100 or even twice that per month.
What do you sell one 8x10 for?  How much money are you losing by being offline?

I would not touch your existing site...instead I would suggest setting up a development site and look at configuring it with the latest coppermine adding the photoshop plugin. And creating you a configurable front page.

Again if I can help let me know,
Gene

[Joachim Müller]

I don't understand how the client gets the full size image

The way he always got it in the past: using coppermine. The tutorial explains how to batch-resize the pics am the client before you upload them. This will result in less resources consumption and therefore your webhost will no longer be mad at you.

I size my images in two different sizes (8x10 and 8x12).

I can tell that you must be a photographer - you think in photo sizes (inches). We're geeks here, we think in pixels, because that's what actually counts on a computer-driven medium.

Someone (Gene-2008) responded to my post. Would you recommend I have him apply this program?

As Gene already said, I hardly know him, so I judge him from my previous experience: he seems to be skilled enough to do the job and more. Think of the "contributor" group description next to his nickname as a quality label.

Just to be sure, this will allow me to have thumbnail and intermediate images displayed, without using up all the resources on the server, but when a client needs to download a full size image (with login and password only), they can do it 24/7? That was the reason I have all these full size images on the server.

urm, what? You need to understand a bit more of the interaction of server and client.

As far as dedicated server, the fact that I am using 140gb's of space, the plans I have found are more than $100.00/month. Can you please forward the few sites you saw for a dedicated server? If you say you have the solution, please tell me how I can proceed. I am not knowledgeable enough to do these things myself...Sorry!

The $50 offer was the one I googled: 1&1. Googling some more I found out that you're right: most offers are around $100. Bytemark (the webhost our website runs on) offers a dedicated at $100 as well - I can recommend their services. I don't know much about the US webhosting market though, as I come from Germany.
However, you might want to consider a virtual dedicated server as well - that should be enough for your purposes and is much cheaper. The bottleneck usually is the webspace: with virtual dedicated servers you don't get 160 GB, but only around 50 GB.
But why don't you ask the webhost you're currently with what they could do for you?

[photoguy1]

Joachim,
Okay, I'm calm now and fully understand the batch re-sizing and how it will solve the issue of the server spikes I'm causing. I will look at the sizing set in Coppermine (which I like) and set it up in either Photoshop or the program you suggested. I re-read some of my comments below and I also went, huh?! Today was a bad day for me!

I speak in 8x10 as far as cropping, because most people that buy from me for personal use order 8x10's. But, I also size by pixels and dpi, depending on the specifications the magazine requests.

As far as you "geeks" and how you think, I want my son to be a "geek" just like you!!!

I've e-mailed Gene and I'm confident he will create a fantastic website for me. It's been over three years of nothing but problems, but I'm looking forward to getting it done this year.

Thank You!

Marty

[Joachim Müller]

The $50 offer was the one I googled: 1&1. Googling some more I found out that you're right: most offers are around $100. Bytemark (the webhost our website runs on) offers a dedicated at $100 as well - I can recommend their services. I don't know much about the US webhosting market though, as I come from Germany.

Try the budget webhoster http://www.server4you.com/us/economy/index.php for cheap dedicated servers.

[thejake420]

Relating to your webhost issues:

Lunarpages has a tendency to get stingy about bandwidth/load on shared servers but their dedicated servers are solid. I have a managed dedicated server with them, and although it costs a fair bit more, I'll attest to the fact that their dedicated server support team (you get a separate contact email for dedicated customers) is absolutely phenomenal at taking care of server problems. The cpanel I have gives me a lot of options.

Lunarpages also has everything you need for CPG (Imagemagick etc)

I've done some HUGE galleries and it's fine on dedicated, just too much of a server load when batch processing/resizing images on a shared server. I imagine most hosts would balk at the load it would put on their "cheap" servers as compared to a dedicated one.

I saw 1&1 mentioned in this thread, but I've had a horrible time with them. Clunky admin interface etc. They're cheap but you'll pull your hair out every time you want to do something. I only have one site with them, and it's basically just a forum so I don't care but they're not so good with CPG in my experience. (Don't recall if they have Imagemagick or not, it's been a while.)

I'd recommend shopping around, taking GauGau's (and others') recommendations into account as well when shopping for a new host, if that's how you end up proceeding.

[Joachim Müller]

The German webhoster Hetzner is offering deidcated servers at 50 Euros per month as well, see http://www.hetzner.de/en/
They have an English webpage as well as a German one, so if you don't mind the time zone difference (that may become relevant when asking for live support), they might be an alternative as well.