create a Pre (ETL) Extract, Transform, load with console c#

Asked

Viewed 56 times

1

I have an activity and I don’t know where to start, I have to create a Pre ETL to consume and transform a .txt follows data:

Price List

The Price List file will arrive with the following name: XBOL_QP_PL_OUT_20181010.txt

Pre ETL should receive this file, and split it in two:

Header: XBOL_QP_PL_OUT_H_20181010.txt Item: XBOL_QP_PL_OUT_L_20181010.txt

Rule XBOL_QP_PL_OUT_H_20181010.txt:

inserir a descrição da imagem aqui

  • File with PIPE tab (|);
  • First position, 1 character, "H" identifier (Header);
  • Keep the same timestamp of the original file;

File should stick to the original nomenclature.

inserir a descrição da imagem aqui

Rule XBOL_QP_PL_OUT_L_20181010.txt:

  • File with PIPE tab (|);
  • First position, character 1, identifier "L" (Lines);
  • Keep the same timestamp of the original file;

File should stick to the original nomenclature.

inserir a descrição da imagem aqui

c#

  • What is your doubt, although with this file I was already in doubt also because the base file is the same as the other files but with different header? what is really to do?

  • 1

    A folder will receive the TXT file that will have the data of the H and L lines in this file, I need to separate in 2 files, one for H lines and the other for L lines and play in another folder, all this in a C console#

  • 1

    And then understood the solution?

  • Thank you very much Virgilio I’ll test now ! I’ll let you know as soon as I’m OK

  • 1

    It worked really well thank you

1 answer

1


Basically reads the main file (with the class Streamreader) and with the command Substring of the line which is of the type string makes a comparison of the first position by finding out if the line is from the H file or the L file and adds in the respective files and adds row by row in the files referring to each new file to be created (with the class Streamwriter), example:

string line;
System.IO.StreamReader file = new System.IO.StreamReader(@"./base.txt");
System.IO.StreamWriter fileH = new System.IO.StreamWriter(@"./baseH.txt");
System.IO.StreamWriter fileL = new System.IO.StreamWriter(@"./baseL.txt");
while ((line = file.ReadLine()) != null)
{
    if (line.Substring(0, 1) == "H")
    {
        fileH.WriteLine(line);
    }
    else
    {
        fileL.WriteLine(line);
    }
}
fileH.Flush();
fileH.Dispose();
fileL.Flush();
fileL.Dispose();

file.Close();
file.Dispose();

Ref.

Another way with System.Linq along with Substring

string[] lines = System.IO.File.ReadAllLines(@"./base.txt");
IEnumerable<string> linesH = lines.Where(x => x.Substring(0, 1) == "H");
IEnumerable<string> linesL = lines.Where(x => x.Substring(0, 1) == "L");
System.IO.File.WriteAllLines(@"./baseH.txt", linesH);
System.IO.File.WriteAllLines(@"./baseL.txt", linesL);

Ref.

Browser other questions tagged c#

You are not signed in. Login or sign up in order to post.