List with large amount of items, excessive memory consumption

Asked

Viewed 271 times

2

In my system I have a List<object>, While using the system I add items to it until I get to a certain point that the system starts to lock, to get very slow. My system can add up to forty thousand items on this list, I never got to that point because it hangs before.

What could be done in this situation?

Note: I would not like to have to use a BD

  • 1

    Look guy are many items and is likely to lock, The ideal is to use a BD that is suitable to receive great data, I even understand that you do not want to use DB but we can not exchange cat for hare right? ... I would recommend an internal DB without external connection and such, just to save the items.

  • 1

    Add in the question a code snippet that demonstrates how this list is being created, the way the question is getting hard to develop a solution.

  • Not a local BD like SQL Compact or Sqlite?

2 answers

2

If you know that it adds up to 40,000 items, it is recommended to put a limit on that list so that you have a performance gain described at that link.

Every time you add an element to the list, and if the list is with its full memory allocation, a new memory block is used twice the current size, and copies everything to this block(performance drop) and keeps adding new elements until the list is full and the process repeats itself.

Then use:

List<object> lista = new List<object>(40000);

ANOTHER POSSIBLE PROBLEM:

If possible, please install the list with defined type, not "Object", but I don’t know if you have any performance gain.

0

Face If you don’t want to Use a BD You can try an approach such as serializing your list into small blocks in XML or Stream Even format, and store it on disk in an organized way by creating a system to search and maintain this data. Or use a local sqlite bank that is exactly what it will do. Because anyway if an object of yours uses 20kb of memory multiplied by the total of the list is what it will consume and will not go out of memory because you keep it in use.

Browser other questions tagged

You are not signed in. Login or sign up in order to post.