Problem reading UTF-8 files in PHP and right in the browser

Question

I need to save a file .txt in UTF-8 in PHP. Saving the file is very simple, however, when opening the file directly in the browser (example www.site.com/file.txt) the browser cannot recognize the UTF-8 encoding and the "Bugam" characters and end up like this:

Having searched the internet, I found some solutions, where people recommend adding GOOD in the file. To add the GOOD is very simple, just use one of these 3 codes.

fwrite($file, "\xEF\xBB\xBF"); 

fwrite($file, pack("CCC",0xef,0xbb,0xbf)); 

fwrite($file, chr(0xEF).chr(0xBB).chr(0xBF)); 

After I added BOM the browser correctly interpreted all the characters and this solved the problem. The end result was more or less like this:

But over time users have reported a problem. The contents of this file can be displayed in two ways on my system. The first way is direct viewing by the equal browser displayed above and the second way is normal viewing on an HTML page. Inside this HTML page there is a <textarea> with the file text and when using shortcuts CTRL + A to select everything and CTRL + C to copy the entire contents of the file displayed in <textarea>, the control character of the GOOD is copied together, and at the time of pasting the copied content the result is more or less like this:

.

As you can see, the first character of the copied text is a '?' which in this case is a control character, which is the equivalent of the GOOD.

My doubt is:

• Is there any way to force the browser to read the file in UTF-8 without the need to add BOM to the file?

OR

• Is there any way to read and display the contents of the file. txt in PHP without the GOOD being shown together?

Answer 1

Answering the two questions.

.

1) Is there any way to force the browser to read the file in UTF-8 without the need to add BOM to the file?

A: Yes, it is possible to "force" the browser to interpret the contents of the text file in UTF-8 by adding the following rule Header set Content-Type "text/html; charset=utf-8" inside the archive .htaccess (apache).

.

2) Is there any way to read and display the contents of the file . txt in PHP without the BOM being displayed together?

A: More or less. There is no native PHP method for reading file contents without BOM. Whenever we use methods such as file_get_contents() or similar, PHP returns ALL the contents of the file, including BOM, so to fix this we can do a "gambiarra" by removing the first character from the String. Example $texto = substr(file_get_contents("arquivo.txt"), 1);

Answer 2

The solution

Create the files without BOM, and configure your web server to serve UTF-8 files by default. What is happening is that it is configured for another encoding, or in case it is not configured, browsers interpret the files as ISO-8859-1, which causes bad impression.

That’s because you’re actually facing two different problems:

1. A UTF-8 file may or may not have BOM

The BOM clearly indicates that a file is in UTF-8, but this is not necessary, apart from not even being recommended, precisely for causing this kind of problem you are having. See: See: https://en.wikipedia.org/wiki/Byte_order_mark

Then one possible solution is to ensure that you are writing a valid UTF-8 file without BOM.

2. You control the web page that displays that file?

If yes, just add the code that eliminates GOOD. After all, it is invalid to appear in the middle of any text. Web page is a text, and the character is probably being displayed after headers, which is invalid (on a text page).

If not, well... point that error at whoever controls the page. Even if it does not control the source/generation of the file, it has to control to never appear the GOOD char in the middle of a text.

Answer 3

To convert a file to utf 8, use the function mb_convert_encoding()