How Do We Really Understand Streams?

Question

Work for a while with C# and . NET and several times I have seen the use of streams out there for reading files, writing HTML response, uploading files and etc. It turns out that I until today did not understand what a stream really is, when they should be used nor how it works.

For example, to read a file we use the following:

using (StreamReader streamReader = new StreamReader(caminhoArquivo))
{
    string conteudo = streamReader.ReadToEnd();
}

Another example is uploading data to a Microsoft Azure blob

using (Stream stream = File.OpenRead(caminhoArquivo))
{
    blockBlob.UploadFromStream(stream);
} 

And another example, which does not involve files, is to write the contents of an HTTP response in an OWIN middleware. Researching a little about streams I found the following:

We have an abstract class that represents a sequence of bytes in which we can perform read and write operations. This abstract class is called Stream.

So a stream is a byte sequence? Even so, when should it be used? Why do we use streams in these cases I mentioned? In the case of the HTTP response is even a little more difficult to understand, we could not only write the response text instead of a stream?

Answer 1

Almost that. Stream is a sequence of data, elements. Can be bytes, the most common, but not necessarily. Individual elements can be more complex or simpler objects, such as the bit. If we were to translate the word, we would probably call it continuous flow. It’s just a concept.

Flow is the key word there. The stream is created precisely to avoid having to deal with the whole. With it it is possible to go handling the desired data on demand.

With the stream the application works with a connection model always. The code starts the connection by opening the stream and may use it until it is closed by code, eventually decided by signaling received during the process. The connection need not be an external mechanism, just have a data collection.

Perks

These data may or may not be available as a whole. It can come/go slowly solving a potential problem of memory overload (of any kind), congestion and temporary absence of transmission. As the stream is open and gives no indication that it has finished it can provide new data even if a wait is required.

Another advantage is that you don’t need to know how the information comes/goes, no matter the source or target. You don’t need to know that the stream works in a certain way. If the data source/target is a disk file, in memory, or object in memory, a network transmission in any protocol, a database connection, a communication with another process, an algorithm that generates this data on demand (random numbers, for example) or otherwise, this is data source/target problem with the stream.

The program that will consume the stream doesn’t need to know how the data gets to it. That’s why it’s called the abstract class (nothing to do with abstract OOP, although this mechanism is used in OO languages to implement it). You don’t need to know the concrete implementation of the data source/target. You also don’t need to know which mode is used for data provision. Several mechanisms can be used within it without the consumer code of the stream need to be aware.

Perhaps a cited example that fits explanation is the generation of random. Generating random numbers does not mean that the generator is a stream. Just like generating a string Nor is it. The stream encapsulates data. Then a random generator can be the data source of stream but not the stream in itself. We cannot call any sequence of elements stream, needs to meet the protocol established by him.

A stream can even manipulate data in certain situations if there is some reason for its mechanism to provide something like this. Still the programmer doesn’t need to know internally how and why he does it, just needs to understand how the data comes and goes through them.

Apis that accept stream

Theoretically if an API wants to provide a way to work with data without the stream no problem. An HTTP response could work directly with the text, but would lose all these advantages cited above. For example, System.IO.File.WriteAllText() does not work with stream.

Obviously if the API only accepts communication through streams you will have to do it this way. No general has only advantages in using this way.

Downside

Perhaps one could say that there is disadvantage by having an extra layer, which slightly increases the complexity and processing consumption. But they are very small things that are not usually relevant to the advantages presented by their use. I will point out that the extra cost of its use is minimal in every sense.

A real example of implementation

Maybe scan the class code Stream can help (or disturb :)) understand how it works. Note that it is only the basis for the classes it actually provides streams that know how to deal with certain sources/targets. It may be interesting to analyze the code of some classes that inherit from it, such as the FileStream.