SiT! Bugs

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0000548SiT!inbound emailpublic2009-03-03 10:452009-03-30 15:11
Reporternicdev 
Assigned To 
PrioritynormalSeveritymajorReproducibilityunable to reproduce
StatusclosedResolutionno change required 
PlatformOSOS Version
Product Version3.45 
Target VersionFixed in Version 
Summary0000548: Importing emails with special characters fails to import correctly
DescriptionWhen receiving email with with special characters, the import is incorrect and the mail is unreadible. This was working for us with the patched 3.41, but in 3.45 it fails indefinitely. The mail imports but badly..
Additional InformationWe sent this as a test:
Subject: [600] test accent
Alors comment ça marche avec des accents : &é »’(-è_çà) ?


and the tracker imported this:

Subject: [600] test accent
Alors comment ?a marche avec des accents : &? ?'(-?_??) ?


TagsNo tags attached.
Attached Filespng file icon bug548-ok.png [^] (96,076 bytes) 2009-03-04 14:58

- Relationships
related to 0000620closedivan Text within < and > is stripped from incoming emails 
related to 0000254closedivan Special international characters in email is shown as ? 

-  Notes
User avatar (0000685)
ivan (administrator)
2009-03-04 15:00

Hi Nicdev,

Again, thanks for the bug report. However... I'm unable to reproduce this, see the attached screenshot.

Are you sure something else didn't alter the character encoding before it hit the database?

Ivan
User avatar (0000716)
nicdev (developer)
2009-03-05 12:28

Hi Ivan,

I am not sure but we did not change anything other than upgrade to 3.45, so i am not sure. This was an old problem that you guys helped with (late one night if i rememeber) and in fact we did a 'half upgrade' to the future svn that would become 3.45, i used the new inbound script and some other includes and it worked very well, but now i does not work at all ... and i really don't want to go back to 3.41 ;-)

Let me know if there is anything i can send you ... to help find fault ...

Nico
User avatar (0000836)
nicdev (developer)
2009-03-26 09:57

Hi Ivan,

It seems we still have these issues, and it is preventing us from upgrading to 3.45. From certain email clients (with certain encodings the characters are completely messed up.

For eg. the original sentence in the mail is:
"Test < alors ça marche avec les accents &é »'(-è_çà)= ?! >"

and we get:
"Test < alors ?a marche avec les accents &? ?'(-?_??)= ?! >"

Please can someone have a look at it, we tried but it is a very complex function....
User avatar (0000839)
kieran (administrator)
2009-03-26 11:01

Can you paste the headers of the emails that are messing up. If the client is sending the wrong encoding (outlook) then there's nothing we can really do.
User avatar (0000840)
nicdev (developer)
2009-03-26 11:30

Hi Kieran,

I will paste some headers in a lttle while. My concern here is that with our "patched" 3.41 the import works. With "patched" i mean it was 3.41 but running the newer inbound script...

If i import the same mails that works in 3.4"patched" they do not work in 3.45.

When i get back to the office i am going to compare the 2 version of inboundmail.php.
User avatar (0000841)
nicdev (developer)
2009-03-26 14:42

Kieran, I sent a mail to Ivan (could not find your email in a hurry), to forward to you with a thread for an email, thus header, sit log info and then what is put in the DB updates.

Let me know if you need anything else..

Cheers
User avatar (0000860)
ivan (administrator)
2009-03-29 11:40

I'm still having trouble reproducing this with the latest svn, I think the problem must be related to the sending client and the charset being used. I did find however that text within < > does get stripped out.
User avatar (0000875)
ivan (administrator)
2009-03-30 15:05

Versions of SiT earlier than v3.31 did not set database collation/encoding to UTF-8 so those people that installed v3.30 or earlier may have incorrect database collation settings (depending on their MySQL defaults).

SiT requires database, table and column collation to be 'UTF8', which it should be for all installations of v3.31 or later.
User avatar (0000876)
ivan (administrator)
2009-03-30 15:11

This script looks useful

http://www.urbannatives.net/localtreechild/drupal/drupal_code/script_converting_your_mysql_database_collation_utf8_and_utf8_general_ci_and_tabl [^]

- Issue History
Date Modified Username Field Change
2009-03-03 10:45 nicdev New Issue
2009-03-04 14:58 ivan File Added: bug548-ok.png
2009-03-04 15:00 ivan Note Added: 0000685
2009-03-04 15:00 ivan Reproducibility always => unable to reproduce
2009-03-04 15:00 ivan Status new => feedback
2009-03-05 12:28 nicdev Note Added: 0000716
2009-03-26 09:57 nicdev Note Added: 0000836
2009-03-26 11:01 kieran Note Added: 0000839
2009-03-26 11:30 nicdev Note Added: 0000840
2009-03-26 14:42 nicdev Note Added: 0000841
2009-03-29 11:40 ivan Note Added: 0000860
2009-03-29 11:42 ivan Relationship added related to 0000620
2009-03-29 12:06 ivan Relationship added related to 0000254
2009-03-30 15:05 ivan Note Added: 0000875
2009-03-30 15:05 ivan Status feedback => closed
2009-03-30 15:05 ivan Resolution open => no change required
2009-03-30 15:11 ivan Note Added: 0000876


Copyright © 2000 - 2020 MantisBT Team
Powered by Mantis Bugtracker