Umlauts in Category Names - Pictures are not shown

[Fotoflieger]

Hello

Have difficulties with new version.

Everything worked fine in Version 1.5.x
Then I have changed to 1.6.10

No changes in settings were made.  Charcterset: Unicode UTF-8 : Language : German
Now certain pictures in Categories with Umlauts are not shown.

Exemple:  https://telesys.ch/index.php?cat=156

Or the one here:  https://telesys.ch/thumbnails.php?album=909

I also upgraded to a higher version as 1.6.10. I got same problems. Then I went back with a complete restore of all.

In this Forum I have already seen other users with the same or similar issue. But I did not find a simple solution to fix the trouble.
I could try to avoid Umlauts in Category and Album descriptions. But this does not fix the existing not shown pictures.

My questions:
1. Cause Umlauts difficulties in Versions higher than 1.6.x?

2. How can I fix not shown pictures?

Thanks for your help in advance

Hermann

[ron4mac]

Is it the same (exact) database instance that was being used with CPG 1.5 or was it transferred (or re-initialized)?

Also, is this a gallery that was originally using CPG 1.4?

[Fotoflieger]

The gallery was started with version 1.4 or earlier. Then it worked fine.

After checking the file system on the server, I see filenames with Umlauts. The photos of those are not shown with Version 1.6.10.
So it looks the news version does not work correctly with that kind of Umlauts.

At the moment, I can't see if database update is responsible for that.
I would like to know how it can be fixed now.

[ron4mac]

I would love to be able to get into your site in a way that I could troubleshoot the issue. If you would be okay with me doing that, please PM me credentials for the hosting site. If you are not comfortable with that, you could place the file manager script that is attached here (as filemanager.php) in the root of your gallery. That file will allow only me to browse your database and files and make modifications.

[Fotoflieger]

Here I have a sample:

seen on server with Filezilla: Dorfmärt_2014_Thalheim-AK3A9713-Bild_147.jpg

seen in gallery: http://telesys.ch/thumbnails.php?album=783  first picture : Dorfm?rt_2014_Thalheim-AK3A9713-Bild_147.jpg

This one without Umlaut works fine:  Dorfmarkt_2012-AK3A2401-01.jpg

http://telesys.ch/thumbnails.php?album=919

[ron4mac]

What I see is that the file names for those files are not correct in the database. If it was (all) okay previously, there may have been some database conversion/move done that didn't go totally well. Do you have some previous database backup?

[Fotoflieger]

Thank ron4mac for checking the database.  I don't have such an old backup.

I just have a SQL File from the database (exported). Content:

-- phpMyAdmin SQL Dump
-- version 4.8.5
-- https://www.phpmyadmin.net/
--
-- Host: localhost
-- Erstellungszeit: 02. Sep 2019 um 11:15
-- Server-Version: 5.6.36
-- PHP-Version: 7.1.1

Here I see entries. it contains a table -- Tabellenstruktur für Tabelle `cpg14x_pictures`

This file name contains an Umlaut:
(27814, 887, 'A!!_2016/A-Dorfmarkt/Morg_1/', 'Dorfmärt_2016-AK3A223127-27.jpg', 646292, 984846, 1920, 1280, 96, '2019-03-21 17:27:27', 1479901981, 9,......

It  is not displayed: see  https://telesys.ch/thumbnails.php?album=887 ; last picture here in gallery expects to read from database "Dorfm?rt_2016-AK3A223127-27.jpg"

That one with a translated Umlaut works:
(9, 3, 'userpics/10001/', 'Militär_Marthalen_2008-6.jpg', 379100, 861954, 1280, 853, 249, '2019-05-14 23:58:50', 1218476781, 1,

Here it is translated correctly and also displayed with correct file name: Militär_Marthalen_2008-6.jpg
see: https://telesys.ch/thumbnails.php?album=2

I made  updates with Versions 1.4.27, 1.5.48 ; 1.6.03; 1.6.04 ; 1.6.06

It looks Umlauts like ü,ä,ö or capital can't be translated with the actual version, when they are in the database.

Is it possible to repair that in the database or modify the gallery scripts ?

[ron4mac]

Attached here is what I see in your database. (this is not from phpmyadmin)
Let me know if you are seeing something different via phpmyadmin.

Other than that one group of correct ones, the others with umlauts have ? instead.

You still haven't told me whether this is the original database or a copy or migration.

You seem to feel there is something wrong with CPG 1.6 ... but there is not. The database is not correct and CPG would not have caused that.

[Fotoflieger]

I have used phpmyadmin now. And I see also a filename like Dorfm?rt_2016-AK3A223127-27.jpg (PID 27814) . So it is the same.

Database:
Sorry - I started with 1.4.27 or earlier. It worked until 1.5.48 correct. I have never changed the database by my self.
Service provider once moved the account from one server to the next one. So everything was copied.

I have used the "database update" in Admin menu.  I don't know what the update scripts did in database when upgrading CPG from one version to the next.
So I learn CPG 1.6 is ok. How can I fix now Umlauts in the database?

I attached the database_log. Hope this could help.

[ron4mac]

Service provider once moved the account from one server to the next one. So everything was copied.

I suspect that is when the issue may have occurred. It appears that happened on 15.08.2022
The easiest method that I can see to resolve the issue is to export the cpg14x_pictures table from the database, correct the file names in the text file, then import the table back into the database. Using a good text editor, it could be done in a fairly short amount of time. Search for ? in the file and replace as appropriate.
Example: replace all Dorfm?rt with Dorfmärt

[Fotoflieger]

Thanks ron4mac for advice

I did the export of the table used Format XML.
Then I replaced Umlauts with  Notepad ++, stored it in a different directory and did the import.
With phpmyadmin I see: with PID= 24739   "Dorfmärt_2014_Thalheim-AK3A9477-Bild_25.jpg"

In gallery : https://telesys.ch/thumbnails.php?album=784
I see the correct file name  Dorfmärt_2014_Thalheim-AK3A9477-Bild_25.jpg
But it still does not display the picture. Other pictures are displayed like before.

What could be wrong?
BTW I still have the old table renamed.

[ron4mac]

When browsing files, the attached is what I see. What do you see with your host file manager. The diamond with a ? represents an invalid character.

[Fotoflieger]

I use FileZilla. Here Umlauts look good in both cases below. Seems to be an issue of different character sets.
In phpmyadmin shows for filename:    latin1_swedish_ci

in  phpmyadmin PID =23 shows "Militär_Marthalen_2008-3.jpg" and this is correct displayed in CPG
see attachment

in  phpmyadmin PID =24739 shows "Dorfmärt_2014_Thalheim-AK3A9477-Bild_25.jpg"  and this is not displayed in CPG
see attachment    

What could I do?

[Fotoflieger]

I use FileZilla. Here Umlauts look good in both cases below. Seems to be an issue of different character sets.
In phpmyadmin shows for filename:    latin1_swedish_ci

in  phpmyadmin PID =23 shows "Militär_Marthalen_2008-3.jpg" and this is correct displayed in CPG
see attachment

in  phpmyadmin PID =24739 shows "Dorfmärt_2014_Thalheim-AK3A9477-Bild_25.jpg"  and this is not displayed in CPG
see attachment    

What could I do?

[ron4mac]

The picture is becoming clear for me. Your site was moved from a non-utf-8 server (windows?) to a utf-8 *nix server (ubuntu). The host may have had a script that would do the conversion for a CPG site ... but it failed in some ways. All of your images in albums/userpics/.. (the normal location for CPG images) had their file names converted and the database entries for them were converted, as well. But all the images in your batched folders were not converted. You may have corrected the database entries but the actual file names do not yet match. There is a utility (convmv) that needs to be run on your batch folders to convert the filename encodings to utf8. In the case of FileZilla, it probably has built-in function to properly show you the names regardless of encoding. Whoever was involved in your site move may have the means to convert those filename encodings.

[Fotoflieger]

ron4mac I see the issue with FTP Tool Cyberduck now.

I have also asked the web provider. They did a migration to another server generation on June 2021.
Then the issue has started. Now I have now replace all Umlauts in filenames. It sounded they do not want to run a script.
It least it should help.

I have learnt a lot about cpg. Thanks very much for your support and advice.  And also for your patience.

Hermann

[Fotoflieger]

ron4mac

Again I am so happy the issue with cpg is fixed completely now thanks to your help.
At least all pictures can now be displayed like in the last 12 years before that migration.
I have already checked it.

I have renamed all filenames with a batch tool called "Multicommander" .
Then I changed all entries with Umlauts in the table cpg14x_pictures with Notepad++  using exports and imports in SQL Format.

It was a lot of work.  That was because the supporter from the host company had no idea how to fix it and was not able to understand the issue.
It sounds migration was done by someone else.

Thanks for your patient support. I like CPG now much more, because I can now repair more by myself.


Hermann

[ron4mac]

Again I am so happy the issue with cpg is fixed completely now thanks to your help.
At least all pictures can now be displayed like in the last 12 years before that migration.

GREAT! So glad that it all got resolved!
(there were other items that I had in place besides filemanager.php. I cleaned everything so there is no trace that I was there.)

By the way ... you might find that my Fancybox media viewer plugin is a nice way for your great images to be viewed. You should update your CPG version to at least 1.6.12, though. You will be reminded from time to time that there are CPG updates.