5
I am trying to create several task (async) they will perform the following logic:
- Parse an Html based on a received url with Htmlagilitypack
- Return a product model after parse
- Insert the Product into the database
- Download the product images
- Mark url as read
Items 1 and 4, especially 4 take time because of the speed of the internet link, so they should be async. But I’m having difficulties, all my code runs, but in a synchronous way.
private static void Main(string[] args)
{
IEnumerable<UrlsProdutos> registros = db.UrlsTable.Where(w => w.Lido == false).Take(1000);
ExecutaTarefasAsync(registros).Wait();
}
public static async Task ExecutaTarefasAsync(IEnumerable<UrlsProdutos> registros)
{
var urlTasks = registros.Select((registro, index) =>
{
Task downloadTask = default(Task);
//parsing html
var produtoTask = ExtraiDados.ParseHtml(registro.Url);
if (produtoTask.IsCompleted)
{
var produto = produtoTask.Result;
//aqui faço um insert com Dapper
downloadTask = InsertAdo.InsertAdoStpAsync(produto);
}
//marca url como lida, igual ao insert do produto
InsertAdo.MarcaComoLido(registro.UrlProdutoId);
Output(index);
return downloadTask;
});
await Task.WhenAll(urlTasks);
}
public static void Output(int id)
{
Console.WriteLine($"Executando {id.ToString()}");
}
The Insert made a fixed just to test
public static async Task InsertAdoStpAsync(Imovel imovel)
{
var stringConnection = db.Database.Connection.ConnectionString;
var con = new SqlConnection(stringConnection);
var sqlQuery = "insert tblProdutos...etc..etc"
con.ExecuteAsync(sqlQuery);
}
I don’t know if each function should be async. or if I could select type the Download and parse to be async..
My async photo download system works perfectly.
public static async Task DownloadData(IEnumerable<FotosProdutos> urls)
{
var urlTasks = urls.Select((url, index) =>
{
var filename = "";
var wc = new WebClient();
string path = "C:\teste\" + url.FileName;
var downloadTask = wc.DownloadFileTaskAsync(new Uri(url), path);
return downloadTask;
});
await Task.WhenAll(urlTasks);
}
I need help to make and understand how Executatarefasasync is really async like the photos I can’t even incorporate into this project.
NOTE: I don’t know if the download of the photos I do there in the parse or I put in this task.
I’ll play your code here, but thank you for the lesson! , very good, very detailed...
– Dorathoto
only a question the Downloaddata, what would be the correct location? in the parse products? or there at the root in the program.Cs in the perform tasks? pq it is the longest and it can only run after parseHtml is completed and returns the product model.
– Dorathoto
@Dorathoto as you said is part of your steps to download based on your parse, I would put a call inside Executatarefaasync(). Since the processing of each image seems to be independent, I would test the difference between running the list in parallel or one by one to see which application would best render.
– Diego Jeronymo