DOMDocument and UTF-8 « Digital Humanities Questions & Answers

Digital Humanities Questions & Answers » Programming

DOMDocument and UTF-8

(9 posts) (5 voices)

Asked 6 years ago by inactinique
Latest answer from inactinique
This question has a best answer.

Tags:

inactinique
Member
Hi,

I'm trying to use an under development plug-in for wordpress that is developped by @clioweb and @stakats from the CHNM. The aim of this plug-in is to take the content of a zotero user's CV and to publish it on your wordpress blog. Sean Takats is using it on his blog: http://quintessenceofham.org/cv/

My problem is the following: I'm French, so I've got accent in my name. See here: http://www.zotero.org/inactinique/cv

When I'm using this plug-in, the accents in my name (and in all titles of my zotero CV page also - temporarily, I just found expressions in French with no accents for those titles) are not properly encoded on my website: http://www.clavert.net/wordpress/?page_id=333

If you look at the HTML code of the zotero CV page, the page is declared as UTF-8 encoded (just as my website). Accents in titles, in this code, are directly typed-in (é, è, etc), but in the rest of the source code, accents are encoded in HTML (é è etc).

So I suppose that the problem is the way DOMDocument and loadHTML in the php code of the plug-in are handling special caracters.

The source code of the plug-in is here: https://github.com/inactinique/scholarpress-vitaware

Does someone have an idea? Please, be quite explicit in your answers - I'm starting with php.

Best regards,
Frédéric Clavert
Tweet this question
Posted 6 years ago Permalink
Conal Tuohy
Member

IReplying to @inactinique's post:

I'm not a PHP expert, either, so I can't suggest a proper fix to UTF-8-encode the Zotero data, but perhaps you could change the encoding declaration on that one page?

Posted 6 years ago Permalink
Anupam Basu
Member

You can slip the UTF-8 encoding explicitly into your file. Change line 27 of your code to the following and it should load correctly.

if($html = '<?xml encoding="UTF-8">' . file_get_contents($path)) {

Posted 6 years ago Permalink
inactinique
Member

Thanks for your answers. Unfortunately, it doesn't work. One more thing:
The bad encoding of the - in this case - "é" on my website's CV page is appearing in the code this way: Ã© - meaning that the special caracters have been wrongly encoded with HTML code.

Posted 6 years ago Permalink
Patrick Murray-John

Try adding this to how you create the DOMDocument at line 29.

$dom = new DOMDocument('1.0', 'UTF-8');

It might be redundant with Anupam's suggestion for how to get the html. I'd try it both with and without that approach. One way or the other I think it'll work.

HTH

Posted 6 years ago Permalink

Stéfan Sinclair
Admin

Best Answer

The problem definitely seems to be in the parsing that happens in loadHTML – I don't think the charset meta tag is being respected by the parser, so the parsing is defaulting to latin1 (as per HTTP specs). One possible hack is to inject a second meta tag before you do the HTML loading:

$path = 'http://www.zotero.org/inactinique/cv';
$html = file_get_contents($path);
 
// now we inject another meta tag
$contentType = '<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>';
$html = str_replace('<head>', '<head>' . $contentType, $html);
 
// now we can continue
$dom = new DOMDocument();
@$dom->loadHTML($html);
echo $dom->saveHTML();

Posted 6 years ago Permalink

inactinique
Member

Replying to @Patrick Murray-John's post:

Hi Patrick,
no I already tried this, it does not work.
Thanks,
Frédéric

Posted 6 years ago Permalink
Anupam Basu
Member

It just worked for me across Firefox, Chrome and Safari. Could you make sure you're not loading from a cache? If it still doesn't work, could you try to capture the raw HTTP response with something like FireBug and post it?

best,
-Anupam

Posted 6 years ago Permalink
inactinique
Member

Hi

Replying to @Anupam Basu's post:

I tried with Firefox on another computer with which I never visited the website. It's really not loading properly. Thanks anyway for your help.

Replying to @Stéfan Sinclair's post:

Your trick just worked Stéfan!

Thanks a lot to all for your help!

Best,
Frédéric

Posted 6 years ago Permalink

RSS feed for this topic

Reply

You must log in to post.