What use is memoryview in Python?

Question

I am reading about arrays in Python and came across a function built in call memoryview. I understood that a difference from lists to arrays is that in the latter it is possible to use the memoryview, while, in the former, no.

For example:

from array import array
lista = [0.0, 1.0, 1.0, 1.0, 2.0, 3.0, 4.0,
         4.0, 5.0, 6.0, 6.0, 7.0, 7.0, 8.0, 8.0]

# Retorna erro TypeError: memoryview: a bytes-like object is required, not 'list'
memoryview(lista)

# Mas isto funciona:
numbers = array('d', lista)
memoryview(numbers)

I figured that memoryview could be a kind of equivalent to the function id which, as I understand it, returns the location in the memory of the object. I was especially tempted by this interpretation because memoryview(numbers) returns <memory at 0x7fe5fca911f0>, in my case. However, id(numbers) returns a completely different value.

So what’s the point memoryview? What is the advantage that the memoryview offers to use arrays instead of lists?

Answer 1

Briefly, in Python, the basic types for manipulating binary data are bytes and bytearray. They, as well as arrays, for example, they are supported by the memoryview, who uses the "buffer Protocol" to allow the access the memory of other binary objects without the need copying.

I confess that this is the first time I’ve come across this in Python, but the concept seems similar to Slices rust. I find it interesting to mention this because this kind of thing is usually interesting when working with "lower level" operations, which is the niche of Rust.

What is the advantage?

The advantage is precisely the speed and the lowest cost for access, since no copy will ever be made for access.

Because of this, you can index and perform Slices without incurring any cost, since, I repeat, there will be no copy.

Understand what memoryview returns as a "lens" that allows you to read the elements directly "looking into memory", which is not even the case with the most "common" Python mechanisms. A slicing in lists, for example, copies references.

As I said at the beginning of the question, memoryview can be seen as an API a little more "low level" python. In most cases, it makes no difference to use a array with memoryview or a regular list. However, there are cases where the cost of copies can actually entail a large additional cost. In such cases, it is ideal to use it.

Why doesn’t it work with lists?

The API memoryview is valid only for objects that implement buffer Protocol, what is not the case with lists.

As the value to be "involved" by memoryview must have each "element" with the same memory size, you are not able to use this with lists, for example, since each element of this structure can occupy a different amount of memory.

In your case didn’t work out because of this: you were using a list, data structure that could potentially have values with varying sizes. But when using an array of floats (as in the second example of the question), it works, since each element of the array is guaranteed to be the size of a float, that is, all array elements occupy the same amount of memory. Therefore, the memoryview will be able to make a clear and performative distinction between each element, allowing access without major costs.