Update content of JSON file

Question

I am trying to manipulate a JSON file using Python but whenever I run the program it erases the JSON file causing me to miss the last change, has how to make it just add to the file?

Filing cabinet:

{
  "PERGUNTA 1": "RESPOSTA",
  "PERGUNTA 2": "RESPOSTA",
  "PERGUNTA 3": "RESPOSTA",
  "PERGUNTA 4": "RESPOSTA",
}

Function that saves to file:

guardar = {pergunta.upper():resposta}
with open('dados.json', 'w', encoding='utf-8') as gravar_file:
     json.dump(guardar, gravar_file,ensure_ascii=False,sort_keys=True, indent=4, separators=(',',':'))

Whenever I run this snippet of code I miss the other questions I had in my file.

I changed the argument from w for a:

try:
    guardar = {pergunta.upper():resposta}
    with open('dados.json', 'a', encoding='utf-8') as gravar_file:
        json.dumps(guardar, gravar_file, ensure_ascii=False, indent=4, separators=(',',':'))
        print('Pergunta guardada!')
except:
      print('Não consegui aprender...')
      aprender()

However, now after executing the above function my JSON is as follows:

{
  "PERGUNTA 1": "RESPOSTA",
  "PERGUNTA 2": "RESPOSTA",
  "PERGUNTA 3": "RESPOSTA",
  "PERGUNTA 4": "RESPOSTA",
}{
  "PERGUNTA 5": "RESPOSTA"
}

And it still makes a mistake because Python didn’t put the , to separate objects.

Answer 1

The problem is that just writing the data at the end of the file is not enough to do what you need.

When writing at the end of the file with the option 'a', you are only writing the content of the new question, ignoring the semantics of the file content (actually the new question should be added to the existing JSON object, instead of being written soon after).

That is, the file already has a JSON object containing several questions. Instead of adding a question to this object, what you did was write another object right after. That’s why the file looked like this:

{
  "PERGUNTA 1": "RESPOSTA",
  "PERGUNTA 2": "RESPOSTA",
  "PERGUNTA 3": "RESPOSTA",
  "PERGUNTA 4": "RESPOSTA",
}{
  "PERGUNTA 5": "RESPOSTA"
}

And if you try to read it again, it will be an error, because this is not a valid JSON (since it is now 2 different objects, instead of just one: the first object has questions 1 to 4, and the second object has question 5).

But from what I understand, what you wanted was a single object containing all five questions:

{
  "PERGUNTA 1": "RESPOSTA",
  "PERGUNTA 2": "RESPOSTA",
  "PERGUNTA 3": "RESPOSTA",
  "PERGUNTA 4": "RESPOSTA",
  "PERGUNTA 5": "RESPOSTA"
}

---

An option to resolve would be to open the file and load all its contents with json.load, thus obtaining the respective object. Then you update this object with the new question and overwrite the file.

But instead of writing directly to the same file, I think it’s best to first write the content in a temporary file, and only in the end, if all goes well, do you rename the temporary file. Thus, if an error occurs while writing, the original file is preserved (as the error will only affect the temporary file). It would look like this:

import json
import shutil
import tempfile

with open('dados.json', 'r', encoding='utf-8') as arq, \
     tempfile.NamedTemporaryFile('w', delete=False) as out:
    # ler todo o arquivo e obter o objeto JSON
    dados = json.load(arq)
    # atualizar os dados com a nova pergunta
    dados["PERGUNTA 5"] = "RESPOSTA"
    # escreve o objeto atualizado no arquivo temporário
    json.dump(dados, out, ensure_ascii=False, indent=4, separators=(',',':'))

# se tudo deu certo, renomeia o arquivo temporário
shutil.move(out.name, 'dados.json')

To create the temporary file, I used the module tempfile. With this, after the code runs, the contents of the file will be:

{
    "PERGUNTA 1":"RESPOSTA",
    "PERGUNTA 2":"RESPOSTA",
    "PERGUNTA 3":"RESPOSTA",
    "PERGUNTA 4":"RESPOSTA",
    "PERGUNTA 5":"RESPOSTA"
}

This method also works for other cases, such as deleting one of the questions, or updating their text (because then the file may be smaller than the original, so only writing at the end would not be the most appropriate action).

---

A detail, its original file nay can have the comma after question 4, otherwise it will be an invalid JSON:

{
  "PERGUNTA 1": "RESPOSTA",
  "PERGUNTA 2": "RESPOSTA",
  "PERGUNTA 3": "RESPOSTA",
  "PERGUNTA 4": "RESPOSTA", <-- esta vírgula está errada, e deve ser removida
}

---

Finally, if the idea is that the file should serve to persist the data, but not necessarily need to be read by a person, consider using the modules pickle or shelve - the latter in particular seems to be more suited to your need. An idea would be to first convert your original JSON to the shelve:

import json
import shelve

# primeira vez, carregar o arquivo JSON e criar o arquivo no formato do shelve
with open('dados.json', 'r', encoding='utf-8') as arq, \
     shelve.open('dados_shelve.json') as novos_dados:
    dados = json.load(arq)
    novos_dados.update(dados)

Then, just update the new file using the file itself shelve:

# da segunda vez em diante, basta ler o arquivo com shelve
with shelve.open('dados_shelve.json') as dados:
    # a linha abaixo já atualiza o arquivo
    dados["PERGUNTA 5"] = "RESPOSTA"

The difference is that the shelve creates binary files, not cute text like the JSON file. But if the idea is just to persist the data and keep updating it, without a person having to read it, this is a good alternative.

Answer 2

I can help you with logic:

Load the file contents

Turn content into JSON

Change the json object in memory

Save the JSON object ('overwriting the file')

this way you won’t have to worry about making your code write the correct syntax because when transforming JSON into string the syntax will be correct

Load the file contents

    arquivo = open("dados.json", "r")

turn the contents of the file into a JSON object

import json
conteudo = json.load(arquivo)

Manipulate the Json Object

... o que vc quiser fazer....

Save the JSON object ('overwriting the file')

arquivo.close() # Fecha o arquivo que estava aberto como somente leitura
arquivo = open("dados.json", "w") # Sobrescreve o arquivo
json.dump(conteudo, arquivo) # Salva o Json no arquivo
arquivo.close() # Fecha o arquivo

Answer 3

Yes it is possible, just change the opening mode to write the file. Using your own code:

with open('dados.json', 'a', encoding='utf-8') as gravar_file:
#resto do código

Note that in this solution the difference is the read mode of the files. We are opening the file to write, using the writing strategy at the end of the file.

The reading mode is the second parameter of the open function, when we use the w, We are opening the file for writing, but cleaning the previous content. When we open using the mode a we are opening the file for writing, but always writing at the end of the file.

+---+--------------------------------------+
| w | limpa o conteúdo e escreve valores   |
+---+--------------------------------------+
| a | abre o arquivo para escrita no final |
+---+--------------------------------------+

To learn more about which file opening methods, I recommend view the documentation.