How to duplicate a table by changing string values to decimal

Asked

Viewed 42 times

2

Good afternoon, you guys.

I am learning SQL and I set out to import a table . csv from the central bank to apply my knowledge. The table consists of four columns:

date entered | name of the currency | value of the currency | quantity of the currency in circulation

The problem is that the quantity came badly formatted (number with semicolons, ex: 5,000,000,000.00 to 5 billion).
The file . csv has 380000 lines, so it is very impracticable to edit manually in the source file. I’ve heard about ETL process, I imagine that maybe it can be solved with this, but as I’m starting my studies in database I have no knowledge about it.

What did I do? By importing, I set everything to string to later change the type of values.

So I have two tables now, the first table would be the original one called circulante1 (columns are all string) and the second table I created called circulante2 (string/string/float/double). I know that the date is not string, but it is so much problem for now that I wanted to solve before the values.

So the only way I thought I could do that (and I thought something was wrong, because it didn’t work) was by using this confusion:

    DELIMITER $$
    SELECT count(*) INTO @linhas FROM circulante1;
    DECLARE @contador INT = 0;
    WHILE @contador < @linhas
    BEGIN
        SELECT data_,nome,cast(valor as float) as valor,cast(replace(replace(qntd,'.',''),',','.') AS DOUBLE) as qntd INTO 
        @data_,@nome,@valor,@qntd
        FROM circulante1
        WHERE PRIMARY = @contador + 1  -- pra ser bem sincero não sei exatamente o que fiz aqui, como não tinha nenhuma coluna auto_increment eu tentei usar isso como o index
        INSERT INTO circulante2 (data_,nome,valor,qntd) VALUES (@data_,@nome,@valor,@qntd) 
        SET @contador = @contador + 1;
    END$$
    DELIMITER ;

Error Code: 1064 You have an error in your SQL syntax; check the manual that Corresponds to your Mysql server version for the right syntax to use near 'DECLARE @counter INT = 0; WHILE @counter < @lines BEGIN SELECT data_,name,ca' at line 1

Basically I have no idea where I went wrong, whether it was logic or a syntax problem. Surely there’s a better way to do that too, if you can point me out I appreciate.

--------------- EDIT 20:47 BRT Guys, Ricardo already answered me, but in case someone can tell me, what I did (despite being inefficient) is wrong where? I would like to understand my mistake to take as learning too

  • 1

    answering your question, it is not possible to use loop structure (while, repeat) commands in a query, this needs to be declared within a stored Procedure, so your code must not be working

  • It makes sense, Ricardo, good score. I will study and try to apply within a stored Procedure! Thank you very much!

  • 1

    that goes for the mysql, other sgbds allow. See also variable declaration, different from sql-server, it is not mandatory to declare all variables starting with @. Another point is that you wouldn’t have to do the select for the variables since it makes no changes to them, could do the select straight to the insert, as in the example of my answer ;) good luck

1 answer

1


If you have a table with the data in text, just make a INSERT with SELECT converting the data:

INSERT INTO circulante2 
SELECT CAST(data AS DATE) as data, 
       nome,
       CAST(replace(replace(valor, '.',''), ',', '.') as DECIMAL) as valor,
       CAST(qntd AS SIGNED) as qntd
  FROM circulante1;
  

Some remarks:

See here an example working: https://www.db-fiddle.com/f/3L6ETu2mZGPX4c4R9jhWuG/0

  • Ricardo, excellent! It worked that way, thanks! I’m going to choose your answer as it really solved everything I had done unnecessarily, but only one detail in case someone replicates this, is the QNTD column which should be decimal or double, so when using your solution I changed the variables you suggested, value to float and qntd to double because int would not support (amount of coins/cells in circulation in Brazil reaches the billions!! :)

  • You evaluated the use of data type DECIMAL or NUMERIC?

Browser other questions tagged

You are not signed in. Login or sign up in order to post.