json_encode returning "Malformed UTF-8 characters, possibly incorrectly encoded"

Question

I’m using the Laravel 3 in a given system.

Sometimes I’m having a problem with the json_encode, returning false in some cases.

In this code, I upload an external page and, with the DomDocument, make a foreach in the meta tags and capture the value of the content, saving in a array.

That one array I use the Response::json of Laravel, that internally uses the json_encode.

Sort of like this:

$url = Input::get('url');
$html = file_get_contents($url);

$dom = new DOMDocument();

@$dom->loadHtml('<?xml encoding="UTF-8" version="1.0"?>' . $html);

$dados = array();

foreach ($dom->getElementsByTagName('meta') as $element) {

    $name = trim($element->getAttribute('property'));

    if (! $name || strpos($name, 'og:') === false) continue;

    $dados[$name] = $element->getAttribute('content');
}

return Response::json($dados);

When I use Response::json, is returning me empty, in some cases.

So I did the following check to find out what was wrong:

$json = json_encode($dados);

   if ($json === false) {
      echo json_last_error_msg();
   }

And he returned:

Malformed UTF-8 characters, possibly incorrectly encoded

I checked the contents of the variable $dados, and she was like this:

Array
(
    [og:title] => **Removido**
    [og:description] => Os Dez Mandamentos: chuva de granizo e fogo � a sétima praga a castigar o Egito
    [og:image] => **Removido**
)   

It seems that the problem is being generated because of this character �.

Does anyone know how to get around this problem?

Updating

I did the test of trying to print the html content with DomDocument, using $dom->saveHTML() and I was returned this mistake:

output Conversion failed due to conv error, bytes 0xE9 0x20 0x61 0x20

Answer 1

Since $data is an array, you can apply the function recursively with array_walk_recursive:

array_walk_recursive($dados, function (&$val) {
    if (is_string($val)) {
        $val = mb_convert_encoding($val, 'UTF-8', 'UTF-8');
    }
});

But making it clear that this is a very ugly scam... It is best to identify which source of these incorrect Unicode characters and treat before they arrive in PHP.

Answer 2

I know it is an old topic but I saw in other places many people still with this problem, as I found my solution here I will leave here my solution, as I identified the field that was giving me problem I applied a solution focused on it, in case it was the return of 'Vendor Name'

$stmt->execute();
    $result = $stmt->fetchAll(PDO::FETCH_ASSOC);


    for ($i=0; $i < sizeof($result) ; $i++) { 
        $tempCnpj = $result[$i]['CNPJ'];
        $tempFornecedor = json_encode(html_entity_decode($result[$i]['Nome_fornecedor']),true) ;
        $tempData = $result[$i]['efetivado_data'];
        $tempNota = $result[$i]['valor_nota'];
        $arrResposta[$i] = ["Status"=>"true", "Cnpj"=>"$tempCnpj", "Fornecedor"=>$tempFornecedor, "Data"=>"$tempData", "Nota"=>"$tempNota" ];
    }

    echo json_encode($arrResposta);

Answer 3

Use md_convert_encoding() in return, following example:

$data = mb_convert_encoding($data,"UTF-8","auto");

$json = json_encode($data);