How to transfer file using python socket?

Question

I’m making a program that consists of passing a file using UDP protocol. The user who runs the file client.py should have files in the root folder of the program in order to transfer to the folder in which the file server.py is. Programs don’t get to shooting errors, just don’t happen to download.

Code from server.py

import socket
from os import write
HOST = '127.0.0.1'  # Endereco IP do Servidor
PORT = 5000            # Porta que o Servidor esta
udp = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
orig = (HOST, PORT)
udp.bind(orig)
#Até aqui é a conexão sendo realizada

#Ouvindo os nomes dos arquivos do client que o server poderá escolher.
while True:
    msg, cliente = udp.recvfrom(1024)
    msg = msg.decode()
    if(msg == 'stop'):
        break
    print(msg)
#input da opção escolhida pelo server   
opt = str(input())
optB = bytes(opt, encoding='utf-8')
udp.sendto(optB, cliente)    

#tentativa frustrada de receber o arquivo do client
while True:
    dados = udp.recv(1024)
    if not dados:
        break
    write(dados)

udp.close()

Client code.py

import socket, os

HOST = '127.0.0.1'  # Endereco IP do Servidor
PORT = 5000            # Porta que o Servidor esta
udp = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
dest = (HOST, PORT)
print('Conectado!')
#Até aqui é a conexão sendo realizada
k = 0 
dirs = os.listdir(".")

#Pegando nome dos arquivos que está na pasta do client e mandando pra o server
for file in dirs:
    i = k
    i = str(i)
    
    msg = '[' + i + '] - ' + file 
    msgB = bytes(msg, encoding='utf-8')    
    udp.sendto(msgB, dest)
    k+=1

#Mandando mensagens pra o server
msgB = bytes ('Escolha uma das opções acima digitando um número:', encoding='utf-8')
udp.sendto(msgB, dest)
msgB = bytes ('stop', encoding='utf-8')
udp.sendto(msgB, dest)

#opção que o server escolheu
opt, server = udp.recvfrom(1024) 
opt = int(opt.decode()) 

#tentativa frustrada de enviar o arquivo
arq = open(dirs[opt], 'rb')
for i in arq.readlines():
    udp.send(i,dest)

udp.close()

Answer 1

TL;DR Try to add a form of Handshake between client and server or add a delay between sent packets

The problem is in the way Udps packets are handled. In your example we have the following requirements:

• Package loss = 0%
• Knowing when to finish uploading or reading an arbitrary-sized file
• Know how to handle full network buffers

Unfortunately, the UDP protocol does not have any of the following, but it is still possible to try to make a file transfer using it (said that the package loss is acceptable and Voce knows how to handle lost packages.)

Package loss

In his example, I believe that Voce you are using localhost or a local network (which improves the situation of lost packages.)

Arbitrary sizes

The transfer problem is due to the absence of a Handshake, in other words, trading on the exchange of information between two participants. A simple solution for this would be: send the number of consecutive packages to be sent, add a time-out in the waiting time for the packages among others.

ex:

"Se o cliente não mandar nenhum pacote em 5 segundos, terminamos"

"Se tivermos um pacote faltando, mandamos outro pedido ao cliente"

"Depois do cliente mandar X pacotes, terminamos"

The method you use to make Handshake doesn’t matter but rather know how to deal with the best and worst cases (package loss, connection loss, wrong package order, etc.)

Network buffer congestion (Bufferbloat)

The Bufferbloat occurs when the network buffer fills up because of the low transfer rate of the server (the server can’t read the packets that fast,) with this, the network has no more space to "store the packets" and the Udps packets that can’t get into the buffer are "lost."

Solutions to this problem would be: asking incoming packages to be re-shipped, reducing customer transfer speed, etc.

Completion

Maintaining integrity in communication using UDP is not easy but can be done. An example would be TFPT (Trivial File Transfer Protocol)

My Solution

Warning: As previously said errors like "file size sent > recvfrom size" can cause errors and unexpected reactions in the program

client py.

import socket
import os
import time

# 3 Inicializando client
print("Configurando cliente UDP")
HOST = '127.0.0.1'
PORT = 5000
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
ADDRESS = (HOST, PORT)
sock.connect(ADDRESS)
sock.settimeout(10.0)

# 4 Enviando arquivos ao servidor
file_paths = os.listdir()
print("Enviando nomes dos arquivos...")
for file in file_paths:
    sock.sendall(file.encode())

sock.sendall("stop".encode())


# 7 Esperando resposta do servidor sobre qual arquivo enviar
print("Esperando resposta do servidor...")
msg = sock.recv(4)
index = int.from_bytes(msg, "little")

# 8 Abrindo arquivo para ser enviado

file = open(file_paths[index], "rb")
file_size = os.path.getsize(file_paths[index])  # Size in bits
pacote_em_kilobytes = 512
pacote_em_bytes = pacote_em_kilobytes * 8

# 9 Enviando o numero de pacotes ao servidor
numero_de_pacotes = (file_size // pacote_em_bytes) + 1
sock.sendall(numero_de_pacotes.to_bytes(4, "little"))

# 11 Enviando pacotes
delay = 0.004
tempo_estimado = numero_de_pacotes*(delay*1.2)

print(f"Enviando {numero_de_pacotes} pacotes ao servidor")
print(f"Tempo estimado: {round(tempo_estimado)} sec")


for i in range(numero_de_pacotes):
    packet = file.read(pacote_em_bytes)
    sock.sendall(packet)
    enviado = f"{int((i+1)*pacote_em_kilobytes)}/{int(pacote_em_kilobytes*numero_de_pacotes)}Kb"
    print('\r'+enviado, end='')
    time.sleep(delay)


# Limpando buffers e sockets
sock.close()
file.close()

server.py

import socket
import time

# 1 Server Setup
print("Configurando servidor UDP")
HOST = '127.0.0.1'
PORT = 5000
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((HOST, PORT))
sock.settimeout(10.0)


# 2/5 Esperando receber lista de arquivos do cliente
print("Recebendo lista de arquivos\n")
files = []
while True:
    data, address = sock.recvfrom(512)
    if data.decode(errors="ignore") == "stop":
        break
    file_name = data.decode()
    print(f"[{len(files)}] {file_name}")
    files.append(file_name)

# 6 Enviando ao cliente qual arquivo baixar
file_choice = int(input("\nQual arquivos receber?"))
while not (0 <= file_choice < len(files)):
    print("Opcao invalida!")
    file_choice = int(input("Qual arquivos receber? "))

sock.sendto(file_choice.to_bytes(4, "little"), address)


# 10 Recebendo numero de pacotes
# Queremos saber em quantos pacotes o arquivo sera mandado
data = sock.recv(4)
numero_de_pacotes = int.from_bytes(data, "little")

# 12 Recebendo pacotes
sock.settimeout(5.0)
file = open(files[file_choice], "wb")
pacote_em_kilobytes = 512
pacote_em_bytes = pacote_em_kilobytes * 8

print(f"Recebendo {numero_de_pacotes} pacotes...")
start = time.time()
for i in range(numero_de_pacotes):
    data = sock.recv(pacote_em_bytes)
    file.write(data)
    porcentagem = f"Baixando... {round((100*(i+1))/numero_de_pacotes, 2)}%"
    # print(porcentagem)
    print('\r'+porcentagem, end='')

tempo_de_download = round(time.time()-start, 2)
print(f"\nO download foi completo em {tempo_de_download} sec")

# Limpando buffers e sockets
file.close()
sock.close()