How to quickly recover a large PHP array?

Question

In the example below, I have a array in PHP with about 128,000 records (Portuguese language entries) that I retrieved from another file, and use in some applications, to compare the indexes as HashMap.

array(128521) {
  [0] =>
  string(3) "aba"
  [1] =>
  string(3) "aba"
  [2] =>
  string(4) "abá"
  [3] =>
  string(7) "ababás"
  [4] =>
  // ...
}

Question

How can I keep this one safe array so that you can quickly recover it from the hard drive without affecting the application’s performance too much? The focus is to make the array for the classes that will use it as soon as possible.

Answer 1

In fact it’s completely unfeasible to do that.

128,000 indices in an array, is a large amount to be manipulated and/or compared.

The ideal is to keep such information in the database and search it based on a minimum limit of parameters.

But I have maybe two solutions:

Physical file

This array be kept as a physical file (which extension you want, in json format)

[EDITED] Idea

Just as a complement, in your case, I think it’s best to create a file for each "letter" of the alphabet.

• a. json
• b. json
• c.json

Unless you compare, include values between the array value, for example: Ada

• Ada
• adá
• guava
• giraffe (joke, haha)

With this I believe that the search, is faster (in a single file), but of course, it will increase the number of requests on the server, and to help in this case, leave any file information inline (), so that the size is reduced/compressed to the maximum.

---

Cached page (json/xml)

I don’t like and/or recommend xml, but feel free

Have a page (route) of your system/site, which contains this array also in json format, however, such page will be cached for the time determined by you.

That way, when other applications go to the specific page (for example: http://www.examplo.com.br/ptbr.json or http://www.examplo.com.br/ptbr.php - the extension itself does not matter, but rather how you will handle the request), you return that page, which in this case will be cached, and will not be loaded again.

---

Doubt

Do you use any framework? The vast majority of them already have a cache system ready, and an excellent administration of the extension/response/protocol type of their request/route.

---

Some links to help you

• Excellent post on cache by Thiago Belem: http://blog.thiagobelem.net/criando-um-sistema-de-cache-no-php/

• Stack Overflow (in English) - search only

• Standard search by google

I hope I’ve helped.

Answer 2

In addition to Nosql options and a layer of Memcached as noted above, a second option would be search engines. This type of problem has characteristics that I would treat with indexing. You will need the entire array or just query certain words according to the search criteria?

If you need the second option I would use a tool like the Apache Solr or Elasticsearch to create a dedicated index (e.g., index containing a tuple [hash / word]). Well-tuned indexes are very fast and already have intelligent internal cache policies, able to return frequent queries "instantly" even in an index with millions of entries.