Output not expected in CSV Python (infinite loop) file

Question

Bom, I’m trying to save the distance and time, result of the difference in the sending/arrival time of the ultrasonic sound, this through the sensor for Raspberry Pi 3 HC-SR04!

After obtaining these values that draw graphs with the Seaborn library, for that precise wanted the data to be written in a CSV file, but this does not happen, the Script is in an infinite loop and always replaces the values in the same line, in the first second the file weighs 15 bytes in the next second already weighs 0 bytes, anyway I can’t get it to write on a new line! I’m new to Python, so I don’t know much about loop techniques or what methods to use!

Requirements:

1. Raspberry Pi 3
2. HC-SR04
3. Python 3

The code is down here:

#Libraries
import os
import RPi.GPIO as GPIO
import time
import csv

#GPIO Mode (BOARD / BCM)
GPIO.setmode(GPIO.BCM)

#set GPIO Pins
GPIO_TRIGGER = 23
GPIO_ECHO = 24

#set GPIO direction (IN / OUT)
GPIO.setup(GPIO_TRIGGER, GPIO.OUT)
GPIO.setup(GPIO_ECHO, GPIO.IN) 

#set CSV
currentPath = os.getcwd()
csv_files = currentPath + "/data_sensor.csv"
csv_columns = ['Time', 'Distance']

def WriteDictToCSV(csv_files, csv_columns, dict_data):
    new_file = False
    try:
        if not os.path.exists(csv_files):
            new_file = True

        with open(csv_files, 'a+') as csvfile:
            writer = csv.DictWriter(csvfile, fieldnames=csv_columns)
            if new_file:
                writer.writeheader()
            for data in dict_data:
                writer.writerow(data)
    except IOError as err:
        print("I/O error({})".format(err))    

def distance():
    # set Trigger to HIGH
    GPIO.output(GPIO_TRIGGER, True)

    # set Trigger after 0.01ms to LOW
    time.sleep(0.00001)
    GPIO.output(GPIO_TRIGGER, False)

    StartTime = time.time()
    StopTime = time.time()

    # save StartTime
    while GPIO.input(GPIO_ECHO) == 0:
        StartTime = time.time()

    # save time of arrival
    while GPIO.input(GPIO_ECHO) == 1:
        StopTime = time.time()

    # time difference between start and arrival
    TimeElapsed = StopTime - StartTime
    # multiply with the sonic speed (34300 cm/s)
    # and divide by 2, because there and back
    Distance = (TimeElapsed * 34300) / 2

    return Distance, TimeElapsed

    if __name__ == '__main__':
        try:
            while True:
                d, t = distance()
                dict_data = {t: d}
                WriteDictToCSV(csv_files, csv_columns, dict_data)
                print('Time Elapsed: {} ms | Distance: {} cm'.format(t,d))
                time.sleep(0.5)
        # Reset by pressing CTRL + C
        except KeyboardInterrupt:
            print("Measurement stopped by User")
            GPIO.cleanup()

Note: Script is not yet complete, more data will be exported and formatted!

For those who do not own an Rpi or HC-SR04, here is a minified version of the code:

import os
import csv

currentPath = os.getcwd()
csv_files = currentPath + "/data_sensor.csv"
csv_columns = ['Time', 'Distance']

def WriteDictToCSV(csv_files, csv_columns, dict_data):
    try:
        with open(csv_files, 'a') as csvfile:
            writer = csv.DictWriter(csvfile, fieldnames=csv_columns)
            writer.writeheader()
            for data in dict_data:
                writer.writerow(data)
    except IOError as err:
        print("I/O error({})".format(err))    
    return

if __name__ == '__main__':
    while True:
        # "d" respresenta a distancia de 12 cm e "t" representa o tempo em ms
        d, t = 12, 0.14732832
        dict_data = {t: d}
        WriteDictToCSV(csv_files, csv_columns, dict_data)
        #print('Time Elapsed: {} ms | Distance: {} cm'.format(t,d))

Output of error is as follows:

Traceback (most recent call last):
  File "/home/pi/Documents/main1.py", line 70, in <module>
    WriteDictToCSV(csv_files, csv_columns, dict_data)
  File "/home/pi/Documents/main1.py", line 34, in WriteDictToCSV
    writer.writerow(data)
  File "/usr/lib/python3.5/csv.py", line 153, in writerow
    return self.writer.writerow(self._dict_to_list(rowdict))
  File "/usr/lib/python3.5/csv.py", line 146, in _dict_to_list
    wrong_fields = [k for k in rowdict if k not in self.fieldnames]
TypeError: 'float' object is not iterable

Answer 1

Sound - every half second this program calls the function WriteDictToCSV that destroys the generated file half a second earlier, and only writes the new data.

The line

with open(csv_files, 'w') as csvfile:

Creates the file again, with 0 bytes, each time it runs.

For such a program, you can leave the file open, and go incrementally recording the data, or open the file in the mode "a" (and not "w"), which opens the file for recording from the end without destroying the previous content.

Try changing the write function of the file to this:

def WriteDictToCSV(csv_files, csv_columns, dict_data):
    new_file = False
    try:
        if not os.path.exists(csv_files):
             new_file = True

        with open(csv_files, 'a+') as csvfile:
            writer = csv.DictWriter(csvfile, fieldnames=csv_columns)
            if new_file:
                writer.writeheader()
            for data in dict_data:
                writer.writerow(data)
    except IOError as err:
        print("I/O error({})".format(err))    

(Data generation is incorrect too - you’re making a lot of confusion with lists and dictionaries, and I recommend doing tests, creating lists and dictionaries in interactive mode, adding values, deleting values, exploring methods - otherwise you probably won’t be able to make any sense of this program)

Answer 2

Its function to write a dictionary does not make much sense.

def WriteDictToCSV(csv_files, csv_columns, dict_data):
    try:
        with open(csv_files, 'a') as csvfile:
            writer = csv.DictWriter(csvfile, fieldnames=csv_columns)
            writer.writeheader()
            for data in dict_data:
                writer.writerow(data)
    except IOError as err:
        print("I/O error({})".format(err))    
    return

You use as a basis the csv.DictWriter, defining the columns ['Time', 'Distance']. This way, the writing function will expect a dictionary in the following format:

{
    'Time': t,
    'Distance': d
}

But what you’re going through is {t: d}.

Also, I see no reason to iterate over the dictionary during writing, as you did in:

for data in dict_data:
    writer.writerow(data)

If dict_data for the dictionary as commented above, just do writer.writerow(dict_data).